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Summary 


This workshop toolkit arose from questions about effective district and school use of data 
posed by education leaders in two Regional Educational Laboratory Northeast & Islands 
(REL-NEI) research alliances who were interested in furthering their work on using data. 
Their questions included: 

• What is a culture of data use? What are its key elements? 

• How can professional development leaders impart the varied skills, knowledge, 
and abilities needed for effective data-use practices? 

• Is there specific guidance for professional development workshops led by district 
and school leaders to improve their culture of data use? 

• What are the right leverage points to start using data effectively? 

• Is there a continuum of practice that educators can follow as their culture shifts 
toward more consistent implementation of data-use practices? 

To help answer these questions, REL-NEI researchers reviewed current research and iden- 
tified five essential elements found in districts and schools with effective data-use practices: 

• Participating in the flow of information for data use. 

• Communicating professional expectations for data use. 

• Providing resources and assistance to make meaning from data. 

• Providing professional development on data-use knowledge and skills. 

• Providing leadership to nurture a culture of data use. 

This guide enables a facilitator to present a conceptual framework aligned with these five 
elements. It also offers professional development materials to support district and school 
leaders in engaging their administrators, teacher leaders, and data team members to 
explore how to establish, maintain, and nurture a culture of data use. 

This step-by-step guide includes an agenda for a one-day professional development session 
(or a series of shorter sessions), guiding ideas to scaffold participant learning, and sugges- 
tions for participant activities. The handouts offer research reviews, vignettes, tools, and 
resources that highlight effective practices in each of the five framework elements. By 
engaging with these resources, participants will clarify their current cultural practices and 
explore strategies for improvement in key framework elements. 
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Introduction 


The Building a Culture of Data Use in Districts and Schools Workshop is geared toward 
administrators, district staff, principals, school data team members, and board of education 
members who make decisions about how data-use practices are developed and sustained 
across their educational institutions. It grew out of the work of two Regional Educational 
Laboratory Northeast & Islands (REL-NEI) research alliances: the Urban School Improve- 
ment Alliance and the Northeast Rural Districts Research Alliance. In their summer 2012 
Research Agenda Workshops, these alliances identified the culture of data use as an area 
in which members needed further training. 

The workshop draws on a conceptual framework of five elements that, collectively, are 
shown in the research as necessary to develop a data-using culture. By engaging in this 
work, and through dialogue grounded in ideas outlined in the Culture of Data Use Frame- 
work case studies and vignettes, participants will apply key elements of this framework 
to their own contexts. The vignettes and case studies allow participants to explore best 
practices and templates for next steps to extend and improve current district and school 
practices. 

The guiding questions are: 

• What does the research say about effective district and school data-use practices? 
What are the essential characteristics of an effective culture of data use? 

• What do districts and schools with an effective culture of data use do? What are 
examples of the essential characteristics in practice? What can participants learn 
from case studies of districts with an effective culture of data use? 

• Which of these practices can be applied in participants’ own settings to deepen or 
extend current practice? 

• What is the role of district and school guidance to support systemic change in 
data-use practices? 

The goals for this workshop are to: 

• Explore the five elements that contribute to an effective culture of data use in 
districts and schools. 

• Learn strategies to extend and deepen the culture of data use in districts and 
schools. 

This workshop is designed to help leaders develop more effective practices, policies, and 
guidance that will support systemic practices to deepen their district’s or school’s culture 
of data use. Following completion of this workshop, participants will have the necessary 
information and tools to lead district or school sessions for developing or improving their 
own culture of data use. 

This workshop does not address strategies for using classroom or school data, such as how 
to collect and analyze data for improvement. For these skills, REL-NEI offers a separate 
workshop entitled Practitioner Data Use in Schools with an associated toolkit (Bocala, 
Henry, Mundry, & Morgan, 2014). 
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Those new to the topic of developing a culture of data use may wish to view REL-NEFs 
online workshop, Building a Culture of Data Use in Schools and Districts, which provides 
background on the culture of data use and presents five elements needed to develop a 
culture of data use (http://workshop-relnei.org/module4ist). In the online workshop, pre- 
senters highlight examples of practices in each element, offer guidance on sustaining key 
practices in each area, and provide an approach for schools or districts to explore their own 
culture of data use. In addition, there are examples of professional development, district 
and school communication, hiring practices, and policy changes that can help shape and 
sustain a culture of data use. 

The workshop described in this toolkit is designed as a one-day, in-person workshop. 
However, it can be structured in various ways, with the caveat that activities should be 
followed in the order that they are presented (as learning is scaffolded throughout the 
workshop). For example, it could take place over one full day, four two-hour sessions, or five 
90-minute sessions. Facilitators and leadership teams can decide the best workshop format 
for their district or school. 

This guide has four parts: this introduction, the facilitator guide, the workshop hand- 
outs, and references. The accompanying slide deck is available at http://www.relnei.org/ 
tools-resources. 

The facilitator guide gives a step-by-step presentation for each segment of the workshop 
to be used in conjunction with the workshop handouts and the slide deck. The workshop 
handouts contain all the tables, case studies, and vignettes used in the one-day session. 
The handouts section can be printed out and distributed at the beginning of the work- 
shop, or sections can be printed by topic. Each topic section and each handout begins on 
a new page for flexible printing. Optional activities are noted in the facilitator guide. The 
slide deck can be customized for individual presentations, and the facilitator guide notes 
which slides need customization. 
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Facilitator guide 


This section provides a general overview on using the guide, suggested agendas, and step- 
by-step guidance on presenting 11 topics on building a culture of data use. 

How to use this guide 


This guide supports local education leaders in facilitating the Building a Culture of Data 
Use in Districts and Schools Workshop in their own settings. It provides background for 
each agenda segment and guidance for each learning activity. 


Throughout this guide symbols are used to highlight key aspects of the material: 


Symbol Meaning 



Important facilitation process note. 



Additional idea a facilitator might be interested 
in raising around a particular topic. 


An essential idea. If this foundational idea is missed, it 
may compromise learning in later activities. 



Materials to be copied prior to the session. 


Some additional helpful hints: 

• Workshop participants should attend in district or school teams (typically five 
to eight members) to help build a supportive network of practitioners engaged in 
developing a culture of data use at their district or school. A district’s or school’s 
data team is the logical choice for who should participate in the workshop. 

• Depending on the size of the district or school, team membership may vary in size 
and role. Examples of data team members include district curriculum and instruc- 
tion directors, district assessment directors, principals, teacher leaders, coaches, 
secondary school department heads, assistant principals, and teachers. 

• These workshop materials can be used in large groups or with one team at a time, 
depending on district or school need. 
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Workshop preparation 



Facilitators should prepare and bring the following supplies and materials: 

• Newsprint or chart paper and easels for group note taking. 

• Large sticky notes. 

• Markers. 

• Copies of worksheets and handouts in this guide for each participant. 

• A computer and projector for the slides, which can be downloaded from 
http://www.relnei.org/tools-resources. 


Handouts 


All handouts are in the workshop handouts section of this guide. They can be copied and 
handed out as a packet at the beginning of the workshop or copied in sections according 
to their agenda topic and passed out at the beginning of the session. 

Pre-reading 


The pre-reading frames the definition of the culture of data use used in the workshop. If 
this workshop is conducted over multiple sessions, the article can be assigned as homework 
between sessions. The suggested pre-reading for the full-day session is at the link below 
and in the workshop handouts section. 

Ronka, D., Geier, R., and Marciniak, M. (June 2010). A practical framework for build- 
ing a data driven district or school: How a focus on data quality, capacity and culture 
supports data-driven action to improve student outcomes . Educational White Paper. 
Boston, MA: Public Consulting Group, http://www.publicconsultinggroup.com/ 
news/post/2010/06/28/A-Practical-Framework-for-Building-a-Data-Driven-District-or 
-School-How-a-Focus-on-Data-Quality-Capacity-and-Culture-Supports-Data-Driven 
-Action-to-Improve-Student-Outcomes.aspx. 

Agenda 


The agenda below details the topics, learning objectives, duration, and corresponding 
slides for this workshop. Modify the agenda as needed to suit local interests. Also modify 
the participant agendas in handout 1.1. 


1 Topic 

Learning objective 

Duration 

Slides 

Handouts 1 

1: Welcome and background 

• Understand the role of Regional 
Education Laboratory Northeast 
& Islands and alliances in 
preparing this content. 

• Welcome and introduce 
facilitators and participants. 

5 minutes 

1-3 

1.1: Participant agenda and 
handouts overview 

2: Workshop overview and goals 

• Clarify what the workshop will 
cover and will not cover. 

• Review the agenda. 

Identify overall learning 
outcomes and success criteria. 

15 minutes 

4-6 
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1 Topic 

Learning objective 

Duration 

Slides 

Handouts ! 

3: Defining a culture of data use 

• Explore the idea that data 
literacy, by itself, is not 
sufficient to ensure data is 
used to guide instruction. 

• Clarify that a culture of data 
use implies that data-use 
practices happen when nobody 
is looking. 

20 minutes 

7-11 

3.1: Definition of a culture of data 
use 

4: Identifying participant and 
presenter assumptions 

• Uncover assumptions that 
individuals bring to this work. 

15 minutes 

12-13 


5: Introducing the Culture of Data 
Use Framework 

• Explain each of the five 
elements in the Culture of Data 
Use Framework. 

• Explore the research that 
undergirds the framework. 

• Understand that all five 
elements of the framework are 
equally important in practice. 

50 minutes 

14-27 

5.1: Culture of Data Use 
Framework: Initial reflections on 
five elements 

5.2: Culture of Data Use 
Framework: Five research 
summaries 

5.3: Culture of Data Use 
Framework: Findings from 
research 

6: Exploring district and school 
practices aligned with the Culture 
of Data Use Framework 

Develop images of practice for 
each element of the framework. 

* Begin to identify a range of 
practices that support work in 
each framework element. 

60 minutes 

28-34 

6.1: Culture of Data Use 
Framework: Vignettes 

6.2: Culture of Data Use 
Framework: Effective data-use 
practices 

7: Considering changing 
expectations of data use 

Identify how education data-use 
practices have been framed in 
recent years. 

• Note current placement on a 
continuum of practice. 

20 minutes 

35-41 

7.1: Changing expectations of 
data use: Teacher activities on 
the continuum 

8: Exploring district data-use 
practices over time 

• Explore through a case study 
how district and school data- 
use practices have changed 
over time. 

• Discuss a range of data-use 
practices that align to the 
framework. 

55 Minutes 

42-45 

8.1: Culture of data use: District 
case study 

9: Reviewing templates, tools, and 
strategies that support consistent 
implementation of data-use 
practices 

• Learn to use each of the 
framework elements to guide 
discussions about current 
practice and next steps. 

* Explore implementation barriers 
and strategies to address those 
barriers. 

45 minutes 

46-56 

9.1: Discussion protocol: 
Identifying implementation 
strategies for a culture of data use 

9.2: Barriers to a culture of data 
use, by framework element 

9.3: Examples of policy and 
guidance to support a culture of 
data use, by framework element 

9.4: Examples of district and school 
guidance to support a culture of 
data use, by framework element 

10: Selecting strategies to 
further implement key framework 
elements 

• Identify areas of focus for high- 
level strategies to consider in 
local settings. 

50 minutes 

57-60 

10.1: Discussion prompts 

10.2: Guiding questions for 
developing policy or guidance to 
support a culture of data use, by 
framework element 

10.3: Examples of policy and 
guidance to support a culture of 
data use, by framework element 

11: Reflection and closing 

• Reflect on new learning. 

• Clarify next steps. 

15 minutes 

61-67 
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Topic 1: Welcome and background 


• Duration: 5 minutes 

• Handout Id: Agenda and handouts overview 

• Slides 1-3 

Facilitator guidance 

Facilitators introduce the workshop and identify why this topic is a useful district or school 
focus- Be prepared to discuss participants’ specific goals and expectations- Slide 3 gives a 
brief overview of how work on establishing a culture of data use in schools originated. 

Presentation slides and participant activities 


Slide 1: 



Explain that this workshop presents a framework for developing a culture of data use in 
which participants will identify their areas of strength and weakness and develop areas of 
focus to improve the culture of data use in their district or school. 

Provide a personalized workshop introduction, including why you selected this topic for 
this group. 


Slide 2: 


Today’s Presenters 

• [Insert Information about facilitator 

[Insert Information about facilitator 

#1] 

#2] 

n pi Regional Educational 

r L “ rya,EDC EM— 

HAIRWestEd* 2 
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Customize slide 2 to provide information about the facilitator. Introduce yourself and allow 
participants to briefly introduce themselves, giving their names and positions in relation to 
education data use. 


Slide 3: 


How were these materials 
developed? 


Network of 10 Regional Educational 


Laboratory (RELs) across the 
country 

Funded by U.S. Department of 
Education, Institute of Education 
Sciences, 2012-2017 
Helping states and districts use 
research and data to inform policy 
and practice with the goal of 
improving student outcomes 



les.ed.gov/ncee/edlabs 


The Culture of Data Use Workshop 
materials developed through REL 
Northeast & Islands (RELNEI) 


REL 

nv « 
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Information to elaborate on slide 3 


Regional Education Laboratory Northeast & Islands (REL-NEI) is one of 10 regional educational 
laboratories across the country charged with helping states and districts systematically use 
data and analysis to answer important issues of education policy and practice with the goal of 
improving student outcomes. The Regional Educational Laboratory Program is funded by the 
Institute of Education Sciences at the U.S. Department of Education. 

REL-NEI builds research capacity and a knowledge base across the region by: 

• Assisting states, districts, and schools in using their data systems. 

• Conducting and supporting high-quality research and evaluation that focuses on four 
regional priorities. 

• Providing opportunities for practitioners to engage with education research. 

• Helping education policymakers and practitioners incorporate data-based inquiry practices 
into regular decisionmaking. 

The Culture of Data Use Workshop materials were developed to support the work of two 
alliances that are part of REL-NEI. Alliance members were interested in learning more about 
what is required to support an effective culture of data use in their districts and schools. 
These educators asked REL-NEI to help them think about what is valuable, beyond preparing 
data sets and providing professional development to build teachers’ and administrators’ use 
of data to support and sustain effective data practices. The Culture of Data Use Workshop 
grew out of this request. 
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Topic 2: Workshop overview and goals 


• Duration: 15 minutes 

• No handout 

• Slides 4-6 

Facilitator guidance 

This section provides a strategic understanding of how to support data-use practices over 
time- In addition, the materials point to key elements of data use in districts and schools, 
so that participants can think about these elements more strategically. Participants will not 
necessarily walk away with a sense of what to “do.” Rather, they will gain an understanding 
of how a framework with five elements can be used to consider and analyze current practic- 
es and to establish ways to better leverage and support educators’ data practices over time. 

The workshop materials are designed to support teachers, teacher leaders, and district and 
school leaders regardless of where they are in their current data-use practices. The work- 
shop goals are organized to help teams or individuals better understand their current prac- 
tice, with respect to the framework, and clarify areas of practice that would benefit from 
additional support. 

This workshop does not focus on developing specific data-use skills, such as using data dia- 
logue, conducting data analysis through an inquiry cycle, or applying specific strategies for 
how to link data to instruction or how to set up a database for storing and accessing data. 

Presentation slides and participant activities 


Slide 4: 


Workshop Agenda Topics 



Review the agenda, noting any modifications. This agenda provides six hours of facilitated 
content. 
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Slide 5: 


Guiding Questions for the Workshop 


What does the research tell us about effective district and school data 
practices? What are the essential characteristics of an effective culture of 
data use? 

What do schools and districts with an effective culture of data use do? What 
are examples of the essential characteristics in practice? What can we learn 
from case studies of districts with effective cultures of data use? 

Which of these practices can be applied in your own setting to 
deepen/extend current practice? 

What is the role of district and school-level guidance to support systemic 
change in data-use practices 
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Raise the guiding questions for this workshop: 

• What does the research say about effective district and school data practices? 
What are the essential characteristics of an effective culture of data use? 

• What do districts and schools with an effective culture of data use do? What are 
examples of the essential characteristics in practice? What can you learn from case 
studies of districts with an effective culture of data use? 

• Which of these practices can be applied in participants’ own setting to deepen or 
extend current practice? 

• What is the role of district and school guidance to support systemic change in 
data-use practices? 


Workshop Overview and Learning 
Goals 


Slide 6: 


Learning Goals: 

• explore five characteristics that contribute to an effective culture of data use 
in schools and districts 

• learn strategies to extend/deepen the culture of data use in your district and 
schools 

Success Criteria: 

• apply the five characteristics of the culture of data to your school or district 
practice - regardless of your stage of development 

• identify high-leverage strategies that you can use to strengthen the culture 
of data use in your setting 

• identify policies or written guidance you can develop that will support more 
consistent implementation of data-use practices 
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Outline the goals and success criteria for the workshop. 

Learning goals 

• Explore the five elements that contribute to an effective culture of data use in 
districts and schools. 

• Learn strategies to extend and deepen the culture of data use in districts and 
schools. 
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Success criteria 


After this workshop, participants will be able to: 

• Apply the five characteristics of the culture of data to their district or school prac- 
tice — regardless of their stage of development. 

• Identify high-leverage strategies that they can use to strengthen the culture of data 
use in their setting. 

• Identify policies or written guidance they can develop to support more consistent 
implementation of data-use practices. 
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Topic 3: Defining a culture of data use 


• Duration: 20 minutes 

• Handout 3d: Definition of culture of data use 

• Slides 7-11 

Facilitator guidance 

This section helps participants understand the definition of this workshop’s key term: 
culture of data use- Because this term is used throughout the workshop, it is important 
to give participants time to explore the definition and compare it to their current under- 
standing of the term- While participants will deepen their understanding throughout the 
workshop, at the close of this section they should be able to share their understanding of 
the term, as used in the pre-reading- 

This section focuses on the intersection of effective data use and culture, with the aim of 
improving the daily practices of educators to support teaching and learning- The definition 
of the culture of data use in the pre-reading (Ronka, Geier, and Marciniak, 2010) provides 
key information about the elements in play in an effective culture of data use- This defini- 
tion begins a dialogue, which will extend throughout the workshop, related to the culture 
of data use, elements of continuous improvement, collaboration, a vision for data use, mod- 
eling, and instructional change- 

The facilitator’s primary role in this session is to help scaffold participants’ understanding 
by moving through three main ideas: What is effective data use? What is culture? And — 
when these two terms are put together — what is the definition of a culture of data use? 
Facilitators can use dialogue prompts to engage participants and further their understand- 
ing of these terms - 

Presentation slides and participant activities 


Slide 7: 


Two Definitions of Data Literacy 


A process that integrates the analysis of 
educational data to support decisions intended to 
improve teaching and learning at the school and 
classroom levels. 

Means, Padilla and Gallagher (2010), Use of Education Data at the Local 
Level , USDOE, page C-3 


Understanding the properties of data and how 
you can use them to inform practice. 


Jeffrey Wayman, 2012 REL-NEI Webinar, The Data-lnformed District, 
Research on Using Data to Inform Practice 
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This section focuses on the intersection of data literacy and culture with 
the aim of improving the daily practices of educators to support teaching 
and learning- Data literacy is a foundational skill for educators, but research 
suggests that data literacy skills alone are not enough to guarantee broad use- 
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Introduce the two definitions of data literacy in slide 7. 

Note that slide 7 addresses question 1 with two definitions of data literacy rooted in using 
data to support teaching and learning. While data are in play at every level of the educa- 
tion system, the materials and resources referenced here focus on student-learning data. 

For example, in the second definition, the term practice refers to instructional practices 
that support teaching and learning. 


Slide 8: 


Defining Data Literacy - Turn and Talk 


With a partner, first discuss these two definitions of data 
literacy. In what ways are these ideas consistent with 
your own definition of data literacy? 


Second, still with your partner, discuss 
the following: 

Is having educators with good data 
literacy skills enough to ensure that 
data is used on a regular basis to 
ensure ongoing improvement in 
student learning? 
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Ask participants to talk to the person next to them using the prompts on slide 8. 

• Are the two definitions of data literacy consistent with your own definition? 

• Is it enough to have educators with good data literacy skills to ensure that data are 
used on a regular basis to ensure ongoing improvement in student learning? 

Provide about two minutes for the first discussion prompt, then about two minutes for the 
second. Responses to the second question may include several topics that will be raised in 
upcoming slides, so hearing those now can help frame the upcoming work. 

Finally, ask a few participants to report to the entire group to hear the kinds of things that 
were discussed in answer to the two questions. 



A question for the groups to consider 


Beyond good data literacy skills, what else did you identify as necessary to ensure that data 
are used on a regular basis? 

Participants are likely to raise these ideas: 

• Leadership. 

• Technology. 

• Access to useful data. 

• Access to instructional data. 

• Collaboration. 

• Time to make meaning from data. 

Make note (on paper or a flip chart) of the topics participants raise. These topics are likely 
to be elements within the Culture of Data Use Framework. 
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Slide 9: 



Note that slide 9 deals with the second question: “What is culture?” 

Discuss this definition of culture: 

• “The way things are done when nobody is looking” (Wayman, 2012). 



Ideas to build on slide 9 and lead into slide 10 


In applying this definition of culture to thinking about what makes a culture of data use, several 
ideas emerge: 

• In the context of data use, culture implies that data use takes place throughout the orga- 
nization in a way that is part of “how business gets done” — in other words, how things 
happen when nobody is looking. 

• In the context of data use, culture then refers to the organizational perceptions and prac- 
tices related to data use. 

• In these environments, data-use practices are not seen as “another thing”; they are seen 

as a tool to get the daily work of teaching and learning done well. 

• In these ways, a culture of data use implies that there is more in place than just data 

literacy skills. 

This workshop looks closely at what is in place in schools with effective cultures of data 
use, so that districts and schools can determine how to establish and deepen a culture of data 
use. 
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Slide 10: 


Definition of Culture of Data Use 


A strong data culture results when an organization believes 
in continuous improvement and regularly puts that belief into 
practice. Schools and districts that have strong data cultures 
emphasize collaboration as a keystone for success and they 
empower teachers and administrators to make decisions for 
which they will be held accountable. Elements of a strong 
data culture include commitment, vision, beliefs, 
accountability, collaboration, modeling, and commitment to 
ongoing instructional and programmatic improvements 

Ronka, Geierand Marciniak (2010) 
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Note that slide 10 addresses the third question: “What is a culture of data use?” 



This definition is crucial for the rest of the workshop and will ground partic- 
ipant experiences as they move forward. Read the definition in slide 10, and 
let participants reflect on this definition as outlined in slide 11. 


As a group activity, call attention to handout 3.1 as you begin to talk about this definition. 
Have participants use the printed definition for the activity outlined on slide 11. 


Slide 11: 


Reflections on the definition of a 
culture of data use 


• In the previous slide we reviewed the 
definition of a culture of data use. 


Individually, write down 3 keywords from this 
definition that align with your understanding 
of a culture of data use. Then, write down 
anything from the definition that is different 
from your understanding. (3 minutes) 
Discuss what you wrote in a small group. Is 
there agreement in your group about the 
definition, or, are there still questions? (5 
minutes) 



• Bring any questions to the whole group for 
review. (3 minutes) 
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Ask participants in their groups to: 

• Identify three keywords that they think are most important to this definition. 

• Write individual responses, then share them with others in the group. 
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Reconvene the large group and have each group summarize its discussions. If time permits, 
use the prompts below for a large group discussion. 



Prompts to deepen dialogue for a large group review of slide 11 


• How is your definition of the culture of data use similar to, or different from, the Ronka, 
Geier, and Marciniak (2010) definition? 

• Which aspects of data use are you most interested in exploring as we work through today’s 
session? 

Participants should now have a sense that “a culture of data use” means that organiza- 
tional systems and structures are in place to ensure this work is ongoing, supports continuous 
improvement, and happens “when nobody is looking.” 
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Topic 4: Identifying participant and presenter assumptions 


• Duration: 15 minutes 

• No handout 

• Slides 12-13 

Facilitator guidance 

This section models presenter assumptions and identifies participant assumptions- 

Individual assumptions about data-use practices, and particularly about developing a 
culture of data use, can greatly impact this work- For example, if an assumption is that 
teachers will use data in the course of their own daily work, it follows that the system 
would need to set up regular, frequent times during which teachers can review data- Simi- 
larly, if an assumption is that teachers need to have time to use data in the course of their 
daily work, then asking teachers to bring this task home, or to do it after school, would not 
align with that assumption. 

If an assumption is that it helps to identify these assumptions, then it helps to begin the 
process of having teams identify their assumptions, so that they can collectively improve 
their approach to this work. 

It is normal for people in different roles to come at this work with different assumptions. 
Thus the agenda allows 15 minutes for facilitators to model the process of raising and 
discussing their own assumptions that may frame the way they approach this work. If more 
time is available, an optional team activity is available. 

First, facilitators can select three or four assumptions from the list below or develop their 
own list to include on slide 12. When sharing these assumptions, facilitators should high- 
light their understanding of why it is important to consider these assumptions. 

For example, suppose the facilitator selected the assumption “Evidence informs profession- 
al judgment; it does not replace professional judgment. Recognize teachers’ wealth of tacit 
knowledge as a starting point.” The facilitator might explain the assumption by saying that 
teachers’ professional judgment is at the heart of data-analysis practices. The purpose of 
the culture of data use is to have more educators using data more often to come to better 
decisions. For this to take place, teachers’ professional judgment will be tapped to explore 
key questions about what inference can be accurately made from the data, and what can 
be done to improve a program, a curricular goal, or an instructional action to support 
improved outcomes. The facilitator might assume that professional judgment is the glue 
that holds this work together — it is what helps move from “data analysis activities” done 
independently from learning to a data-using culture, which is what happens with data 
“when nobody is looking.” 

This assumption was offered as a starting point for dialogue. Other assumptions from this 
list are equally effective starting points and may speak more to the facilitator’s background, 
beliefs, or experiences. 
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Assumptions that relate to developing a culture of data use include: 

• There is a tension between compliance-driven data use and inquiry-based data 
use (for example, where teams examine data on a specific instructional issue and 
take time to explore multiple ways to solve the issue) that impacts the framing of 
a culture of data use. The more accountability driven, the potentially greater the 
challenge to build norms of inquiry and to create a safe environment necessary for 
effective data use. 

• The goal of having a data-using culture is that everybody’s practice improves. 

• Interpreting evidence is not a solo act; meaning comes from how a variety of indi- 
viduals at different levels of the education system understand and make sense of 
data. 

• Evidence informs professional judgment; it does not replace professional judgment. 
Recognize teachers’ wealth of tacit knowledge as a starting point. 

• Data do not, by themselves, lead to improvement. 

• Educators need professional development on instructional decisionmaking that 
considers the role of data. 

• Culture of data use is built only when you set up the structures and practices, not 
the other way around. 

• The context, the setting, and the environment in which data are delivered all 
matter. 

• Personnel in effective data-using cultures use data in the course of their work, not 
in addition to their regular work. 

• It is essential for educators at all levels of the education system to build multiple 
fluencies regarding data use. This is a key underpinning of a data-using culture. 

Presentation slides and participant activities 


Slide 12: 


Assumptions underlying today’s 
work: 

• [Facilitators select 3-4 assumptions from the list provided in 
the facilitator’s guide, or, write your own. These will be used 
as discussion starter for assumptions participants bring to this 
work.] 
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List your selected assumptions on slide 12 and share them with the group. Explain why you 
think your assumptions are correct. Allow participants to challenge your assumptions. 
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Slide 13: 


Reflections on “assumptions” 


Turn and talk with a neighbor about the information 
shared on the previous slide. Do you have any questions 
about the assumptions listed? 


Reflect on the list of assumptions with your partner. 
Discuss how an assumption raised has been true (or 
not) in your own experience? 

In the large group, hear 
from a few participants about 
their understanding of the 
assumptions listed. 
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Optional exercise for connecting participants with their own 
assumptions 


If more time is available, ask participants to select one of the assumptions in slide 12 with 
which they agree or disagree. Show slide 13 and ask participants to discuss with a neighbor 
their experience with one of the assumptions — does it hold true in their experience? Provide 
about three minutes for the partner discussion and two to three minutes for a large group 
discussion. Listen to two or three reports, then do a quick summary. Keep in mind that this is 
a quick review. If complex ideas or disagreements arise, note them on a flip chart for review at 
a later time. 
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Topic 5: Introducing the Culture of Data Use Framework 


• Duration: 50 minutes 

• Handout 5.1: Culture of Data Use Framework: Initial reflections on five elements 

• Handout 5.2: Culture of Data Use Framework: Five research summaries 

• Handout 53: Culture of Data Use Framework: Findings from research 

• Slides 14-27 

Facilitator guidance 

This section introduces the research that undergirds the Culture of Data Use Framework. 

The Culture of Data Use Framework stems from work in districts and schools over the 
past 15 years to support implementing various kinds of data-use practices — including 
training data coaches, supporting implementation of professional learning communities, 
leading content-based professional learning related to data use, and supporting leadership 
coaching. This work yielded five elements necessary for establishing data-use practices that 
become cultural norms. 

The initial design of the framework put leadership as the foundation for the four other 
elements (figure 1). Along the right side, the focus is on teachers’ learning new skills and 
knowledge: the top right panel describes making meaning from data through collaborative 
inquiry skills, and the bottom right panel addresses deepening content knowledge and 
skills to use the data. Along the left side, the topics are structured around communication: 
the top left panel addresses the importance of establishing systems that ensure access to 
usable data, and the bottom left panel covers communicating a vision of effective data use, 
with a focus on how these practices will support instructional improvements (and develop 
teacher capacity) over time. 


Figure 1. Culture of Data Use Framework: Initial reflections on five elements 


Ensuring access to data 

Data are accessed, coordinated, filtered, and prepared in ways 
that allow educators to quickly and efficiently analyze and 
interpret data to answer key questions and address important 
teaching and learning issues. 


Making meaning from data 

Protected time is provided to allow users of data to collectively 
make sense of what the data indicate and to explore how to 
move from data to evidence that will inform instruction. This time 
is focused on making meaning from data and is supported by the 
use of consistent inquiry-based practices. 


Clarifying expectations for data use 

There are clear expectations about how to use data, and these 
expectations change over time as skills for data use grow. 
District, school, and teacher leaders frame consistent messages 
about how data can be used to support teachers’ professional 
capacity and student learning. 


Building knowledge and skills to use data 

Adequate professional learning takes place regarding data 
use, assessment literacy, and using data to inform instruction. 
Professional learning is integrated into daily practices and 
supports teachers in building content knowledge over time. 
Professional learning is differentiated to support teachers’ 
specific learning needs. 


Leading a culture of data use 

Leadership nurtures and supports a culture of data use and develops organizational structures that include time and resources to 
conduct ongoing data dialogue and feedback that will support users to act on new knowledge. Acting on knowledge is supported at 
the administrative, teacher, and student levels. Leaders’ use of data is central to helping educators interact around issues that will 
lead to improved learning outcomes. 
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This framework is intentionally neutral in relation to specific interventions designed to 
improve data culture, and it does not attempt to address how an effective culture of data 
use is developed- There are several reasons for this- First, there is not yet enough knowledge 
from research about which interventions, under what circumstances, are most effective 
(Hamilton et aL, 2009; Marsh, 2012)- Second, many districts and schools are not set up 
to provide specific interventions- For example, if a school is not able to create a schedule 
that includes structured time for data team meetings, it is a shared responsibility, working 
with the school leadership team, to identify how to organize time for data analysis in other 
ways- Time might be provided during early release days, within faculty meetings, at depart- 
ment meetings, or in a “data room” (a room at the school where teachers can post, discuss, 
and give feedback on student data)- While these intervention methods might not meet the 
full measure of continuous work in data analysis, with the right structures and protocols, 
they can be effective in helping schools move forward in their data-use practices. 

After outlining this initial framework, 60 studies were reviewed on developing a culture of 
data use — studies that focused beyond just data literacy and looked at the range of prac- 
tices necessary to support effective use of educational data at the district and school levels. 
Five studies in particular focused on naming the practices required for this work- These 
studies are outlined in slides 20-22. 

In this section participants will be exposed to the five elements of figure 1 and, in an 
“each teach” activity (described below), will review and share key ideas that undergird each 
element. This activity provides a cursory review of the Culture of Data Use Framework 
with the goal that participants understand that each of these five elements is fairly well 
understood in the literature. 

It is not necessary at this time to address all questions related to this framework- More 
work will be done with the framework through the course of the workshop. At this point, 
the introduction to each element is sufficient. 

Presentation slides and participant activities 



Show slide 14 (same as figure 1 and handout 5-1), which gives an overview of the initial 
understandings of five major areas of work necessary for data use to become systemic and 
to happen “when nobody is looking.” This slide shows the five elements together, four of 
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them resting on the base of leadership. The elements are pulled apart in slides 15-19 for a 
closer look. 


Review slide 14, pointing out how the ideas are organized: 

• Entry points at the school level are often through the elements on the right side. 
The top right “Making meaning from data” panel focuses on developing collab- 
orative inquiry. Just below, the “Building knowledge and skills to use data” panel 
focuses on ensuring teachers are supported to explore and deepen their content 
knowledge and focus on pedagogical data literacy (or the ability to apply data to 
instruction in a specific content area). 

• Along the left side, the primary focus is communication. The top left panel, 
“Ensuring access to data,” focuses on the importance of establishing systems that 
provide access to usable data. Just below, the “Clarifying expectations for data 
use” panel relates to communicating how data-use work supports instructional 
improvements and helps build teacher capacity over time. 

• The bottom, or foundation, is leadership. Leadership of data practices is required 
to support the four other elements of the framework. 


Briefly highlight each element in slides 15-19. There will be time during the “each teach” 
activity outlined in slide 20 for participants to discuss each of these five framework 
elements. 


Slide 15: 


Initial outline of a Culture of Data Use 
(Figure 1, Element 1) 

ENSURING ACCESS TO DATA 


Data is accessed, coordinated, filtered and prepared in 
ways that allow educators to quickly and efficiently 
analyze and interpret data to answer key questions and 
address important teaching and learning issue. 
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Initial outline of a Culture of Data Use 
(Figure 1, Element 2) 

MAKING MEANING FROM DATA 


Slide 16: 


Protected time is provided to allow users of data to 
collectively make sense of what the data indicate and to 
explore how to move from data to evidence that will 
inform instruction. This time is focused on making 
meaning from data and is supported by the use of 
consistent inquiry-based practices. 
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Slide 17: 

Initial outline of a Culture of Data Use 
(Figure 1, Element 3) 

CLARIFYING EXPECTATIONS FOR DATA USE 

There are clear expectations about how to use data, and 
these expectations change over time as skills for data 
use grow. District, school and teacher leaders frame 
consistent messages about how data can be used to 
support teachers’ professional capacity and student 
learning. 
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Slide 18: 

Initial outline of a Culture of Data Use 

(Figure 1, Element 4) 

BUILDING KNOWLEDGE AND SKILLS TO USE DATA 

Adequate professional learning takes place regarding 
data-use, assessment literacy, and using data to inform 
instruction. Professional learning is integrated into daily 
practices and supports teachers in building content 
knowledge over time. Professional learning is 
differentiated to support teachers’ specific learning 
needs. 
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Slide 19: 

Initial outline of a Culture of Data Use 

(Figure 1, Element 5) 

LEADING A CULTURE OF DATA USE 

Leadership nurtures and supports a culture of data use 
and develops organizational structures that include time 
and resources to conduct ongoing data dialogue and 
feedback that will support users to act on new 
knowledge. Acting on knowledge is supported at the 
administrative, teacher, and student levels. Leaders’ use 
of data is central to helping educators interact around 
issues that will lead to improved learning outcomes. 
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Slide 20: 

“Each Teach” Activity: 

Research Summaries by element 

• Organize into table groups of five 

• Each person at the table is assigned one of the five 
research summaries 

• Each person individually reads their summary and 
identifies 3-4 points to share with their four colleagues at 
the table (10 minutes, quiet reading and reflection) 

• Each person shares the key points from their reading, 
going around the table, until all five research summaries 
are shared (10 minutes, 2 per summary) 

• If possible, try to limit discussion about the general 
framework, as that will take place during the next activity 

Bit ±::r EDC ^ 20 
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Organize groups of five people. Call attention to handout 5.2. Assign each person in the 
group to read and share information from one of the five research summaries. 

The goal of this 20-minute activity is for participants to gain a grounding in the kinds of 
research that support this framework and to begin to form some pictures of best practice in 
each framework element. 

Note that the five research summaries are designed to help shape an under- 
Jf standing of best practices in each area. Taken from studies and research syn- 
theses, the summaries focus on the best of what can be done in each of these 
areas. Ultimately, this workshop can clarify what is possible based on exam- 
ples of where this work is going well. More examples of best practice will be 
provided through the workshop. This exercise is meant to prime the pump 
and to spark a dialogue about a range of things that are effective within each 
element. 


Reconvene the large group and return to the slides. 


Do not have a large group discussion about what each group has done. This will come after 
the next group activity. The goal is for this dialogue and the dialogue in the next activity 
to focus on what is known about highly effective practices. 


Slides 21-23 move the discussion from the research that supports each element of the 
framework to the research that supports the entire framework — all five elements needing 
to be in play at the same time, which is discussed in the next section. For these slides, 
simply share the big idea: 



There is an emerging consensus that the five elements discussed here are 
consistent with what researchers have found as they have looked at schools 
that are more effective at using data in a way that is ongoing and part of the 
culture. 


Slides 21-23 point to different ways that researchers have framed these topics. 



More on framework elements 


For those interested in learning more about framework elements, Jeffrey Wayman’s 2012 
webinar for the REL-NEI Urban School Improvement Alliance (http://www.relnei.org/events/ 
event-archive/the-data-informed-district.html) has a good overview of these elements based 
on his team’s research. 
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Slide 21: 


Emerging Consensus - 

key elements of data-using culture 


Research with consistent findings: 

• Hamilton, et al. (2009) 

- Name four areas of focus for schools: Establish a clear 
vision for data use, develop and maintain a district-wide 
data system, make data part of an ongoing cycle of 
instructional improvement, and provide supports that 
establish a data-driven culture within the school. 


• Park and Datnow (2009) 


Name three elements for data use: Invest in data 
management systems, build capacity to interpret and act 
on data through professional development and facilitation, 
and define goals and establish a culture of data use. 
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Read slide 21, stressing the words that correspond to framework concepts: vision, data 
system, cycle of improvement, data-driven culture, invest in systems, build capacity, define 
goals, and culture of data use. 


Slide 22: 


Emerging Consensus - 

key elements of data-using culture 


Research with consistent findings (continued): 


• Wayman, Jimerson and Cho (2010) 

- Identify five elements to support effective district data 
practices: Collaboration, common understandings, 
professional learning, principal leadership and data 
systems. 

• Honig and Venkateswaran (2012) 

- Name four elements to support consistent districtwide use 
of data: Participating in the flow of information, providing 
resources to help teachers make sense of data, 
communicating professional expectations for data use, and 
providing school-based professional development. 
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In slide 22, note the phrases: “collaboration, common understandings, professional learn- 
ing, leadership, systems” and “flow of information, resources, make sense of data, commu- 
nicating expectations, professional development.” 


Slide 23: 


Jeffrey Wavman’s REL Alliance Webinar (201 2) 


Example of emerging consensus 

• Common Understandings (“Agreeing on which data to use and how 
to use it”) 

• Data Systems (“...which provide useful and accessible access to 
information”) 

• Collaboration (“Collaboration is the life blood of school data-use”) 

• Professional learning (“To develop expertise in data use among 
individuals and the organization”) 

• Principal Leadership (“Data use lives and dies in the principal’s 
office”) 
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There is emerging consensus in the research, which is highlighted in several studies- Way- 
mans language is most closely aligned with the Culture of Data Use Framework used in 
this workshop- His quotes on each line were shared during the REL-NEI webinar in 2012. 


Slide 24: 


A Culture of Data Use Framework 


Participating in the flow of information 
for evidence-use 

Provide resources and assistance to 
make meaning from data 

Communicate professional 
expectations for data use 

Provide professional development on 
data use knowledge and skills 

Provide leadership to nurture a culture of data use 
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The five elements of the Culture of Data Use Framework that will be used throughout 
the remainder of the workshop are highlighted in this slide- The titles of each element of 
the framework have been changed to reflect how researchers have addressed these five key 
findings- Though the researchers’ language is quite different, the main ideas are consistent 
with what we have observed and what district and school leaders identify when addressing 
the important elements of a culture of data use- Take time to review each of the elements 
above. 



This would be a good time to stop for questions about the five elements and 
about the specific attributes of each element. 



Additional guidance for slide 24 


A question that often comes up at this point is why is the framework organized this way? 
Why is it not set up to show interrelationships among the various elements? The answer is 
that researchers know only that all these elements are important, but they do not yet fully 
understand the interrelationship among the five elements. Answers to questions about where 
to start, or what sequence to follow, are not yet clear. But it is clear that these five elements 
contribute to effective practice. 
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An illustrative point 


Slide 25: 


1. Balance 

2. Flexibility 

3. Poise 

4. Strength 

5. Endurance 
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Point out that as a gymnast must train simultaneously in the five elements noted on slide 
25, districts and schools must also attend to all five elements in the framework at the same 
time- Gymnasts who develop one element over the others do not perform as well, overall, 
in competitions- Thus they must work each day to address each of the five elements of 
practice. This is a helpful way to think about district and school support: What is being 
done each day to support each of the five elements? 



All five elements of the framework are equally necessary and must be attend- 
ed to simultaneously. 

• Districts may have focused on only one or two elements, or some elements 
may be underdeveloped, which may affect other elements. For example, 
lack of access to data negatively impacts teachers’ collaborative inquiry. 

• The ability to identify which elements are more or less developed at a par- 
ticular district or school is a skill that should come out of this workshop. 


Slide 26: 


A Culture of Data Use Framework 
Research Summaries - Handout 5.3 


Participating in the flow of information 
for evidence-use 

Provide resources and assistance to 
make meaning from data 

Communicate professional 
expectations for data use 

Provide professional development on 
data use knowledge and skills 

Provide leadership to nurture a culture of data use 
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As you show slide 26, refer to handout 5.2. Point out that slide 26 shows the research ideas 
that support the five elements of the framework. 
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Slide 27: 


Culture of Data Use Framework 

• Are there any surprises in 
this framework? 





- If so, what stands out 

y •" 


that is surprising? 



• Which ideas shown on this 



framework are familiar? 

You Look Familiar 
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Use the prompts on slide 27 to guide discussion of handout 5.2. A long discussion is not 
necessary at this time; just take a moment to review and explore the main ideas discussed 
so far — that each of the five elements are necessary and that each is fairly well understood 
in the research. 

• Are there any surprises in this framework? If so, what stands out that is surprising? 

• Which ideas in this framework are familiar? 



Transitioning to district and school practices 


It might be helpful to know that many people, when seeing this framework for the first time, 
report that these ideas are familiar and that they have personal experience with some of them. 
Having people think about this briefly will help frame the transition into the next activity, in 
which participants will engage with some examples, or vignettes, that capture key practices in 
each framework element. 
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Topic 6: Exploring district and school practices aligned 
with the Culture of Data Use Framework 


• Duration: 60 minutes 

• Handout 6.1: Culture of Data Use Framework: Vignettes 

• Handout 6.2: Culture of Data Use Framework: Effective data-use practices 

• Slides 28-34 

Facilitator guidance 

To use the Culture of Data Use Framework, participants need to have a sense of what prac- 
tices look like in each of the five framework elements. This section has participants explore 
vignettes that highlight examples of practices in each of the five elements. It responds to 
one of the guiding questions for the workshop: 

• What do districts and schools with an effective culture of data use do? What are 
examples of the essential characteristics in practice? 

Nearly all of this section is allocated to reviewing vignettes, which takes place in a “jigsaw” 
activity. A jigsaw provides a streamlined efficient mechanism for participant learning and 
dialogue. The activity involves two rounds of discussion with small groups formed in dif- 
ferent ways. In round 1, “expert” groups are formed, with each examining one of the frame- 
work elements. In round 2, “teaching” groups are formed, each with one person from each 
of the five round 1 groups. Each “expert” member reports the findings from his or her round 
1 group. To conduct this jigsaw, a minimum of 10 participants are required to make five 
expert groups and two teaching groups. (For workshops with fewer than 10 participants, 
individuals can read each vignette and then share their findings, or an individual can read 
a vignette and share key points as described below, in the guiding questions for round 1.) 

Consider the following timing for this activity: 

• 5 minutes for introduction and setting up the jigsaw groups. 

• 20 minutes for expert group reading and review of main points. 

• 20 minutes for teaching groups to form and discuss the findings from each expert 
group member. 

• 15 minutes to review slides 30-34, highlighting handout 6.2, and debrief partici- 
pant learning. 

The goal is to explore the framework elements in the examples of practice in schools that 
have developed successful data-use practices. Keep the dialogue focused on the vignettes to 
help participants think about best practices. Later, participants will have time to explore 
their own practice. 

Presentation slides and participant activities 



During the jigsaw activity, help the groups keep track of time. Ensure that 
groups identify two or three key points during round 1, and that they move 
through discussion of all five vignettes during round 2. 
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Vignette Review - Directions for 
Round 1 of the Jigsaw activity 


Slide 28: 


Participants will be grouped into five (Round 1) discussion groups. 
Each initial group will read and discuss some vignettes that show 
examples of ONE element of the framework. 

During Round 1 , each group will read their assigned vignette for a 
particular element, and discuss what about that vignette exemplifies 
the framework element under review. They will identify 2-3 key 
points to share in Round 2. 

There will be 20 minutes to read, discuss, and identify 2-3 key points 
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Form participants into five groups. These initial groups will be called the expert groups. 
Assign each expert group one element of the framework. 

Call their attention to handout 6.1. Each vignette deals with one of the framework ele- 
ments. Ask each group to read and discuss the vignette corresponding to the framework 
element it was assigned and to develop two or three key points about how the vignette 
exemplifies the framework element. 

Offer the groups these guiding questions for round 1: 

• How did this vignette exemplify the element of the framework? 

• What are examples of practice that illustrate this element of the framework? 

• How did this practice contribute to building a culture of data use? 


Slide 29: 


Vignette Review - Directions for 
Round 2 of the Jigsaw activity 


During Round 2, each table group of five will consist of one expert in 
each framework element. 

Each person has 2-3 minutes to share the key points that were 
identified during Round 1. 

Following discussion of all five vignettes, participants can begin to 
discuss the differences among the five elements of the framework. 
Consider the following discussion prompt: 

- What do you notice about each specific element of the framework? How 
are the concepts in each of the five elements unique? 
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Reorganize the groups so that there is one “expert” on each framework element in each 
group. The groups for round 2 are the “teaching” groups. 

In the new groups, each expert shares two or three key points about how the vignette 
exemplified his or her framework element as raised in round 1. 

Next, group participants discuss the differences among the five elements. Consider these 
discussion questions: 

• What do you notice about each element? 

• How are the concepts in each element unique? 
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Following round 2, use either of the following two methods for a 15 minute debrief of the 
activity: 

• Use slides 30-34 to show key points that were likely raised in the vignettes- On 
the left of each slide is a quote from the research that captures an important idea 
for that framework element- On the right of each slide are examples of practice 
from the vignettes- These slides provide a quick summary of the key points. 

• Conduct a more general, large group debrief of the jigsaw activity- To structure 
this debrief, listen for key points that come up during the group discussions and 
ask participants to share those points with the large group, or ask for general com- 
ments through the following prompts: 

o What did groups discuss during the round 2 dialogue that helped them see 
how each of the five framework elements is unique? 
o Where did groups notice overlap or wonder about how these ideas intersect or 
are dependent on each other? 



Debriefing from the “jigsaw” activity 


A large group debriefing discussion focused on the second prompt about overlaps among the 
five elements might move dialogue forward around the topic of how the framework elements 
are interrelated. All five elements are necessary, and attending to each requires specific knowl- 
edge and practices. Continued examples throughout the workshop should help participants 
better understand how to approach each element to improve their culture. A goal of this work- 
shop is to help participants better understand how to tease these concepts apart for more 
clarity about how to move forward in each area. In other words, understanding each element 
helps participants understand the specific knowledge and skill development that must take 
place in each element. 


Slide 30: 


I. Participating in the flow of information - 
main messages 


“A good data system is not 
enough: Use of data to inform 
instruction requires 
leadership and system 
realignment. Data-driven 
decision making requires 
leader initiative to align 
curriculum and assessment 
practices, professional 
development and data 
systems.” (IES, 2010) 



Participating in the flow of 
information: 

• Create systems that 
support timely, credible, 
short-cycle assessment 
data 

• Create systems to support 
high quality data entry 

• Note shifting role of 
technology and educational 
leaders (from the systems 
to the learning) 

• Ensure teacher feedback is 
built into the system 



For element 1, point out the research finding on the left and the examples from the 
vignettes on the right. Ask participants if they found any other examples- Use the same 
technique on slides 31-34. 
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Slide 31: 


2. Provide resources to make meaning from data- 
main messages 



Slide 32: 


3. Communicate professional expectations 
for data use - main messages 


Central office leaders built a 
“common interpretation and 
orientation towards data 
driven decision making which 
were mediated at the school 
level by formal and informal 
leaders” (Park and Datnow, 
2009). 


Communicate Professional 

Expectations: 

• Build in opportunities to co- 
develop consistent messaging 
across buildings 

• Write clear policies around how 
data will be used, and by whom 

• Include expectations for how 
professional learning will 
support new data analysis 
expectations 

• Model expected practices, 
including use of the inquiry cycle 
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Slide 33: 


4. Provide professional development on data 
use knowledge and skills - main messages 


‘Embedded, informal professional 
learning provided as part of 
teacher collaboration or 
coaching sessions, and 
focused on instructional 
actions informed by evidence, 
appears to offer the strongest 
support for teachers use of 
data in deciding what and how 
to teach.” (IES, 2010) 


Professional learning 
includes: 

• Content knowledge 
(standards, 
curriculum) 

• Pedagogical content 
knowledge 

• Data literacy 

• Assessment literacy 

• "Pedagogical data 
literacy" 

(Mandinach, 2009) 


REL 
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5. Provide leadership to nurture a culture of 
data use - main messages 




“In schools where principals 

District and school 


intentionally employed strategies 

level leadership 


to support data use to improve 

includes: 


student learning, teachers 



typically reported better attitudes 

• Lead Learners 


and more effective data use. In 

• Modeling 


schools where principals 

• Creating safe 


employed fewer or negative 

conditions for 


strategies, attitudes were worse 

analysis 


and data use was reported to be 

• Access to data 


less effective.” (Wayman, Cho 

• Accountability AND 


and Jimerson, 2012) 

support 

& 
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If you debrief using slides 30-34, close with a brief (2 minute) discussion for the large group. 
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Topic 7: Considering changing expectations of data use over time 


• Duration: 20 minutes 

• Handout 7.1: Changing expectations of data use: Teacher activities on the continuum 

• Slides 35-41 

Facilitator guidance 

This section briefly addresses the shift in data-use practices as districts and schools move 
away from an accountability-based approach for teacher review, toward an approach that 
includes student learning data. Participants will reflect on where they see their district or 
school on this continuum and use this knowledge later in the workshop as they explore 
appropriate next steps. 

This frame will help participants explore the culture of data use district case study in topic 
8, which shows how this continuum took shape over multiple years in one district. 

Presentation slides and participant activities 

Since the beginning of the standards movement, there has been an increase in the use of 
accountability assessments and a corresponding increase in the use of large-scale data to 
guide education decisionmaking. Over the past several years, there has been a shift toward 
considering a variety of data types — formative, interim, and benchmark — to inform and 
support teaching and learning. 

Slides 35-37 show characteristics of the shift from an accountability focus to a learning 
focus. 


Slide 35: 
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Slide 36: 


Changing expectations of data use 



Accountability characteristics are shown on the left (tail) of the arrow. Data practices that 
involve a strong accountability focus rely primarily on large-scale, aggregate data. This 
shift reflects a stronger emphasis on gathering useful individual data on student learning, 
collecting the data more frequently, and ensuring that teachers have access to and can use 
the information. 


Slide 37: 



An accountability focus involves a heavy emphasis on information management, or the 
collection and dissemination of organizational information related to student assessment 
results. The shift is toward supporting knowledge management, or thinking about ways 
districts and schools can best support developing knowledge and skills to support data use 
at the classroom level. 


Slide 38: 


Changing expectations of data use - 
teacher activities on the continuum 



planning using student grouping, instructional changes 

accountability data, and attention by based on how to 

and placement teachers on what teach - Entry into 

decisions to teach (ex. re- continuous Material adapted from Mear 

teach groups) improvement padnia and saiiagher, 2010 
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38 
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Slide 38 is adapted from research by Means, Chen, DeBarger, and Padilla (2011), who high- 
light this continuum of practice and the types of activities that might be seen at each of 
these three levels. 

In stage 1, school improvement teams and school leaders use primarily state assessment 
data to inform decisionmaking. State assessment data might inform curriculum decisions, 
teacher placements, and student placements. Often, during this stage there is heavy reli- 
ance on state assessment and accountability data, in part because of limited access to other 
types of data. In this stage 1 there is very little use of data to guide instructional decisions 
or classroom actions. 


In stage 2 there is initial use of other types of data, and data users apply data decisions 
to focus on what to teach. Data are reviewed to see whether specific standards have been 
met and to determine how subgroups are performing. Data-use strategies focus on analy- 
sis of curriculum and standards to ensure that key standards are being addressed and to 
determine whether specific students or student groups should be supported with additional 
content or instructional time. In this stage there is often a new reliance on benchmark or 
interim assessments. The overarching focus is on what is being taught. 

In stage 3 the primary focus is on instructional practices — on how to teach. Teachers are 
using data more frequently, and additional data are available to provide information on 
current student performance. Data analysis strategies are focused on looking at assessment 
data to determine which instructional actions benefited students the most and on explor- 
ing ways to improve instruction based on these outcomes. There is an increasing focus on 
more “formative” classroom data, which are seen as a tool to support daily teaching and 
learning. 


Slide 39: 


Changing expectations of data use - 
teacher activities on the continuum 



STAGE 1 

Focus of work on 

improvement 
planning using 
accountability data, 
and placement 
decisions 


STAGE 2 

Focus of work on 
curriculum 
development, 
student grouping, 
and attention by 
teachers on what 
to teach (ex. re- 
teach groups) 


STAGE 3 

Focus of work 01 
comparative analysis 
of student growth, 
instructional changes 
based on howto 
teach -Entry into 
continuous 
improvement 


RELr 


ESS 


STAGE 4 
Focus on linking 
assessment and 
data literacy at the 
classroom level to 
address how 
students learn, 
with immediate 
feedback based on 
data 



Stage 4, addressing how students learn, links assessment and data literacy in 
the classroom and provides immediate feedback to the teacher. At this point 
on the continuum, the purpose of using data has shifted from compliance to 
helping students learn. 


In stage 4, teachers use student achievement data formatively to understand what students 
know and can do through daily classroom assessments. Stage 4 work involves daily focus 
on student-learning data, especially on what students know and what they need to address 
in the next steps in instruction. The data analysis skills from stages 1-3 are evident in 
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stage 4 but are increasingly focused on understanding the evidence and making real-time 
instructional adjustments to best meet student-learning needs. 


Slide 40: 


Changing expectations of data use - 
teacher activities on the continuum 



teachers on what 
to teach (ex. re- 
teach groups) 


teach - Entry in, 

continuous 

improvement 


Mb 
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students learn, 

with immediate 
feedback based or 
data 


Stage 4 requires a data-using culture. The Data Quality Campaign (2009) points out that 
the shift from compliance to a student-focused culture of data use involves a much greater 
focus on supporting people rather than “things.” Support from district and school leaders 
becomes increasingly important to help educators move into stages 3 and 4* 


Slide 41: 


Individual reflection 

□ Take a moment to 

individually think about 
which stage or stages best 
represent your current 
practice. 



□ What might be some 
preliminary thoughts you 
have about next steps, 
based on your individual 
reflection about current 
practice? 


1133 i~ SAIR UJestEd 9 




To close this section, ask participants to reflect on where they are along this continuum. 
To keep this time “safe,” do not press participants to talk. Individual reflection is a way 
for them to begin to think about their practice and to frame some very preliminary ideas 
about their next steps. 
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Topic 8: Exploring district data-use practices over time 


• Duration: 55 minutes 

• Handout 8d: Culture of data use: District case study 

• Slides 42-45 

Facilitator guidance 

This session explores a case study of how a suburban school district developed a culture 
of data use over a decade- Participants will explore how the district moved along the 
continuum introduced in topic 7 and discuss what steps were taken and what stage was 
reached- The case study is a realistic blend of good practices and setbacks- A key point is 
to understand the amount of preparation and commitment it takes to progress along the 
continuum- 

Participants will work in groups of three to review the case study and then explore ques- 
tions about the changes in district practice over time. A goal is for participants to have a 
dialogue that relates to key aspects of implementation: 

• How data-use practices develop over time, and how district and school leaders can 
support that development. 

• How district and school leaders can consider and respond to the changing needs 
of teachers as they adopt new data-use practices. 

• How to identify and address barriers to consistent data-use practices. 

This case study is not an exemplar for best practice- Rather, it is a real example in which 
the district did some things very well and had some missteps along the way (which is fully 
expected in this work). 

Because the case study is long, this activity has been set up as a “trio reading.” Each person 
in the trio reads a section of the case and shares it with others. 

This section concludes with a large group debriefing that builds on the dialogue in the 
small groups. The debriefing question raises the idea that it can take districts and schools a 
long time to move toward full implementation of a culture of data use. Participants may or 
may not agree as to whether the case district made it to stage 4 implementation (there are 
valid points on both sides). The larger idea is to help participants explore the case study 
to understand how long this work takes and to plan how to continue to support teachers 
over time. This key point is made after the final slide. Additional discussion questions and 
activities using the case study are presented at the end of this section. 

Presentation slides and participant activities 

To set up this activity, organize participants into groups of three. Make sure that each 
group of three has space to work, as it will be difficult to concentrate if other groups are 
very close by (for example, at the same table). 
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The total time is 55 minutes, sequenced as follows: 

• 5 minutes to review the initial slides and set up groups. 

• 20 minutes to read and identify three to four key points to share. 

• 15 minutes to share and discuss ideas about how the district moved through the 
four stages in the continuum. 

• 15 minutes for large group debriefing. 


Slide 42: 


Case Study Review 


The purpose of the case study is to look at an existing 
set of district data practices and to think about how 
this district has addressed developing and supporting 
data use practices over time. 

This case study is not an exemplar for best practice. It 
is, rather, a real example, one in which the district has 
done some things very well, and, also had some mis- 
steps along the way (which we fully expect in this 
work). 


REL " 
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Refer to the case study, Handout 8.1: Culture of data use: District case study, and describe 
the purpose of the assignment. 


Slide 43: 


As you review the case, think about how this 
district has moved through these four stages 



STAGE 1 

Focus of work on 

improvement 
planning using 
accountability data, 
and placement 
decisions 


STAGE 2 

Focus of work on 
curriculum 
development, 
student grouping, 
and attention by 
teachers on what 
to teach (ex. re- 
teach groups) 


REL i 
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STAGE 3 

Focus of work 01 
comparative analysis 
of student growth, 
instructional changes 
based on how to 
teach - Entry into 
continuous 
improvement 
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STAGE 4 
Focus on linking 
assessment and 
data literacy at the 
classroom level to 
address how 
students learn, 
with immediate 
feedback based on 
data 

A- 13 


Review the four stages and note that participants will learn how a district passed through 
these stages over a decade. 
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Slide 44: 


Case Study Process 

• Work in groups of three 

• Everyone reads the introduction page to the case study 

• Each person is assigned one of the following: 

- 2005-2006 and 2006-2007 

- 2007-2008 and 2008-2009 

- 2009-2010 and 2010-2011 


Once each person has read their section 

- Report out in chronological order 

- Share 3-4 key points per year 


- Discuss and agree on 1-2 points that capture the districts’ stage 
of development in each of the three time frames under review. 
Include evidence from the case study that supports your claim. 



EH Sr"‘ SAIR UJestEd 9 



44 


Describe the process of the case study activity in slide 44. Working in groups of three, par- 
ticipants should read the time spans indicated. Each person reports in chronological order, 
and the groups discuss and agree on one or two points that capture the district’s stage of 
development in each timeframe along with supporting evidence. 


Slide 45: 


Debrief-Case Study 


What does this case study 
indicate about the timeframe for 
establishing a culture of data 
use? 


In what ways is the timeframe in 
the case study similar to, or 
different from, the timeframe for 
developing a culture of data use 
in your district? 


What might this case study imply 
about future considerations 
related to the time needed for 
implementation? 
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After the activity, debrief the large group with the three questions in slide 45. 

What does this case study indicate about the timeframe for establishing a culture of data 
use? 

In what ways is the timeframe in the case study similar to, or different from, the timeframe 
for developing a culture of data use in your district? 

What might this case study imply about future considerations related to the time needed 
for implementation? 
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Additional optional questions are below. 



Consideration of timeframe is a significant takeaway for participants. Devel- 
oping a culture of data use takes years and involves consistent attention and a 
focus on addressing the next steps. Leaders have shared again and again that 
this work takes far longer than they anticipated and often involves a long- 
term approach to resource allocation and professional development in ways 
that were not considered in the early stages. Dialogue related to this issue can 
be helpful at any stage, with notable value in the early stages. 



Debriefing from the case study 


This case study can also lend itself to dialogue about a number of issues, from specific exam- 
ples of practice to exploring issues of district and school interactions around data use. The 
following ideas may come up naturally in group dialogue. Depending on district or school next 
steps, these prompts may also be useful during the debriefing: 

• What structures were put in place over time to support teachers in using data? 

• What leadership competencies did the district work to develop? What are some examples 
of these? 

• What faculty and staff competencies did the district work to develop? What are some 
examples? 

• What barriers did the district face in building a culture of data use? How did the district 
overcome those barriers? 

• An area of lesser focus for this district was the framework element, “Participate in the flow 
of information for evidence use.” What are some examples of that from this case? What 
impact might this have had on their overall work? 

• In this case, what worked well in one year but did not when tried again later? What is the 
takeaway from this experience? 
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The optional activity below helps teams take the ideas they learned in the case study and 
apply them to their own setting. This activity involves developing a case study for their 
own school or district. Once done, this written reflection of data practices can be a useful 
tool to clarify work done in the area of data use, identify gaps, and plan next steps in devel- 
oping a culture to support data use. 



Optional activity 


• Create a bulleted list identifying data practices at your school or district over the past five 

years. Use the case study as an exemplar. Involve as many people as you can to recall the 
different district- or school-wide data-use practices introduced overtime. This is best done 
as a group but can also be done independently and then brought to a team meeting and 
shared. 

• Once the bulleted outline is done, map it to a blank Culture of Data Use Framework 

template. Have group members identify which items on the list go in which “box” of the 

framework. 

• After aligning case study elements to the framework, ask the following questions: 

° Which of the five characteristics are more fully attended to? 

° Which have been less attended to? 

° What might be some next steps as you see the information organized in this way? 
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Topic 9: Reviewing templates, tools, and strategies that 
support consistent implementation of data-use practices 


• Duration: 45 minutes 

• Handout 9.1: Discussion protocol: Identifying implementation strategies for a 
culture of data use 

• Handout 9.2: Barriers to a culture of data use, by framework element 

• Handout 9.3 Examples of policy and guidance to support a culture of data use, by 
framework element 

• Handout 9.4 Examples of district and school guidance to support a culture of data 
use, by framework element 

• Slides 46-56 

Facilitator guidance 

Topics 9 and 10 address the final set of guiding questions for this workshop: 

• Which of the practices highlighted in this workshop can be applied in your own 
setting to deepen or extend current practice? 

• How can you create or better support systemic change in data-use practice using 
district- or school-level guidance? 

Participants will explore materials in one framework element of their choosing. The process 
involves working through a discussion protocol while reviewing three workshop handouts. 
Participants will identify one framework element and then review barriers, discuss exam- 
ples of guidance and policy for the elements that other districts and schools have used, and 
highlight how ideas from those examples can be applied within their own setting. 

Depending on the number of participants, the work in this agenda topic can be structured 
to provide opportunities to review all five framework elements. For example, with 20 par- 
ticipants, five groups of four could each discuss one of the framework elements and then 
report their findings. Facilitators can identify what might work best in their setting. 

A primary goal of this section is to familiarize participants with the tables and examples 
of the culture of data use so that these resources can be used beyond this workshop. While 
it is beyond the allotted timeframe to have every participant engage with every handout, 
data teams from districts or schools can easily use the discussion protocol to work through 
each of the five framework elements, ultimately identifying areas of focus or next steps in 
all five elements. 

The structure of this topic is designed to provide a scaffold that helps individuals or team 
members develop, revise, adapt, or reconsider district and school guidance in light of what 
they have learned in this workshop. The debriefing of this section is structured to have the 
large group come to agreement about the value of improved documentation and guidance 
in developing a culture of data use. The next agenda item (topic 10) addresses this work. 
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Presentation slides and participant activities 


Culture of Data Use Framework 


Slide 46: 


Participating in the flow of information 
for evidence-use 

Provide resources and assistance to 
make meaning from data 

Communicate professional 
expectations for data use 

Provide professional development on 
data use knowledge and skills 

Provide leadership to nurture a culture of data use 



Use slide 46 to remind participants that this section is focused on using the Culture of 
Data Use Framework. This section examines barriers to implementing the elements and 
how to develop guidance and policies to foster each element. 


Slide 47: 


Case Study: High-leverage policies 
clarify the shift 

• Provide information about the structure(s) and intended 
use of data systems (I) 

• Guidance to clarify how to use scheduled time available 
to analyze data (II) 

• Outline expectations and structures for learning 
opportunities for teacher data use (III) 

• Outline professional development goals and alignment to 
available supports (IV) 

• Clarify the role of teacher leaders (IV) 


• Set expectations, professional learning and support for 
principal data use (V) 
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Slide 47 shows examples of high-leverage policies that the case study district (in topic 8) 
considered as it moved toward developing a culture of data use. The roman numerals show 
which Culture of Data Use Framework element corresponds to these policies. Use this 
slide to introduce the examples that will be highlighted in this segment. The goal is for 
participants to begin to frame effective practices from the workshop to apply in their own 
setting. 


Slide 48: 


Activity: Review of school and district 
practices by framework element 

Working in small groups of 3-5 people use the discussion 
protocol to: 

• Identify one element of the framework to focus on for 
the next 40 minutes 

• Brainstorm barriers to implementation in that specific 
element of the framework 

• Review Handout 9.2: Barriers to a culture of data use, by 
framework element 

• Review Handout 9.3: Examples of policy and guidance to 
support a culture of data use, by framework element 

• Identify 1-2 high priority ideas to consider in your setting 
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Use slide 48 to introduce the activity for topic 9. Each group selects one element of the 
framework, brainstorms barriers to its implementation, and identifies one or two high-pri- 
ority areas of that element for its home district or school Groups will have 40 minutes 
to work on this activity. Throughout this time, visit each group, support group processes, 
and provide guidance on materials. Times for each question are listed on the discussion 
protocol. 


Slide 49: 


District- and School-Level Guidance 


What written guidance can do: 

- Address inconsistent implementation 

- Help alleviate/address barriers to a culture of data use 

- Clarify the changing roles and interrelationships 
necessary to do this work 

- Support breaking down silos 

- Provide a vehicle to model new forms of collaborative 
work necessary for a culture of data use 

- Other (your ideas) 
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Bring the groups together and use slide 49 to debrief the activity and discuss why having 
written guidance or policies is important to developing a culture of data use. Ask partici- 
pants to add their own ideas about why this is important. 

Consider using this slide as a prompt for large group dialogue about the value of this work 
in increasing understanding and commitment to establishing more consistent practices. 
As explained in the vignettes and research summaries, written guidance and policies help 
districts develop clear, consistent messages about data use and support cultural changes 
toward more effective inquiry and data-use practices. This is a useful time to revisit this 
point. 
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Debriefing the discussion protocol activity 


Slides 50-55 are optional debriefing slides for this agenda segment. They provide a chance 
to discuss or review the examples of policy and guidance more deeply. Using these slides to 
debrief may be more useful with groups that are new to this work, are beginning to form a 
vision of what is possible, or are beginning to discuss a range of potential practices. Show 
the slides in conjunction with handout 9.4 to show participants actual examples of written 
guidance for each element. 

To streamline this review, consider using these slides to highlight only one example per 
element or to ask participants who worked in each framework element to capture what strategies 
they discussed at their tables. Note that slide 56 offers an activity to debrief this in more detail. 


Slide 50: 


Slide 51: 


Handout 9.3 

Examples of policy and guidance 


Participating in the flow of information 
for evidence-use 

Provide resources and assistance to 
make meaning from data 

Communicate professional 
expectations for data use 

Provide professional development on 
data use knowledge and skills 

Provide leadership to nurture a culture of data use 



Handout 9.3 

Examples of policy and guidance - Element 1 


Participating in the flow of information for evidence-use 

Written expectations for data use that show which practices align with 
improved practice and document the shift away an accountability-based 
data use approach. 

Written clarification identifying which users of data are meant to answer 
which questions. 

Calendars and timelines of district data-use expectations by grade-level. 
Description of district data systems to clarify functions and uses. 
Description of the district data system that describes how it supports and 
aligns with the everyday work of educators. 
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Also call attention to handout 9.4. 


Slide 52: 


Handout 9.3 

Examples of policy and guidance - Element 2 

Provide resources and assistance to make meaning from data 

• A school calendar documenting scheduled time to analyze evidence. 

• A common drive with written protocols for using evidence during team 
meetings. 

• Written role expectations for data teams with specific team-member 
functions outlined. 

• A schedule with opportunities for calibration. 

• Agreed upon norms for data-use posted and reviewed at all meetings. 

• Written documentation of how coaches or instructional specialists will 
support collaborative data use practices. 


REL 
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See sample guidance in handout 9.4: 

• Common formative assessment tuning protocol. 

• Collaborative assessment project: Professional learning team member roles. 

• Professional learning team common formative assessment review: Team data analysis 


protocol: Five-phase data review. 



Handout 9.3 

Examples of policy and guidance - Element 3 


Communicate professional expectations for data use 

Slide 53: 

• Written guidance about the focus of data-use as designed to support 
all students. 

• Written communication highlighting how the focus on all students 
will raise the achievement of struggling learners through increased 
differentiation and personalized student supports. 

• Written expectations about annual "products" that include student 
evidence, including written guidance for use of evidence at parent- 
teacher meetings, portfolios, and information that is documented 
across years. 

• * 
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See also in handout 9.4: 

• District guidance on organizing scheduled data reviews. 

• Planning for a coherent assessment system. 


Handout 9.3 

Examples of policy and guidance - Element 4 


Provide professional development on data use knowledge and 
skills 

Slide 54: 

• An annual schedule of professional learning for data-use, including 
formal, informal, large-scale, team-based, and daily learning, focused on 
common learning goals schoolwide, and including individual (or team) 
areas of focus for teacher learning. 

• Written structure to document teacher learning goals regarding using 
data. 

• Protocol for principals to review teachers practices with various types of 
data-use (including both Schoolwide and classroom uses), and to outline 
next steps in their individual learning. 

. 1 
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See also in handout 9.4: 


• Rural middle school data team (embedded professional learning). 

• Data literacy and assessment literacy toolkit topics. 


• Role description of data coach. 


Slide 55: 


Handout 9.3 

Examples of policy and guidance - Element 5 


Provide leadership to nurture a culture of data use 

Job descriptions that capture the role of data-use leaders throughout 
the district - principals, teacher leaders, data team leaders, district 
leaders. 

A hiring protocol outlining expectations or activities that show facility 
with data use during the hiring process. 

Documentation related to how leaders learn data use, with increased 
expectations outlined overtime. 
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See also in handout 9.4: 

• Example of input to district policy: Changing principal job description. 

• Building leadership team development activity to establish agreed upon actions for princi- 
pals to strengthen data-use practices. 
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Slide 56: 


Strategy Review - 
Stand up and share 


Returning to the discussion protocol, each person finds 
two others in the room who were not at their discussion 
table, and has five minutes to discuss the following: 

- Identifies the element under review 

- Notes 2 barriers that were discussed 

- Reviews 2 high leverage strategies in their element 

- Identifies 2-3 points to bring back to the next level of 
discussion 
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Use slide 56 for a group of 10 or more. For small teams, no debriefing is required. In a 
larger group, this dialogue can help quickly share some key points. It offers participants 
a chance to get feedback on what they have discussed and to check it against how others 
approached the same topics. This brief sharing gets participants ready to move into the 
next agenda item. 
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Topic 10: Selecting strategies to further implement framework elements 


• Duration: 50 minutes 

• Handout 10*1: Discussion prompts 

• Handout 10.2: Guiding questions for developing policy or guidance, by framework 
element 

• Handout 103: Examples of policy and guidance to support a culture of data use, by 
framework element 

• Slides 57-60 

Facilitator guidance 

Participants will select one or two framework elements and craft guidance to support con- 
sistent implementation and practice. Once education leaders see the examples in these 
materials and discuss other effective practices (either from their own experience or from 
readings), this work flows quite easily. Writing guidance while the ideas are still fresh from 
the workshop is helpful. Remind participants that there is no road map or manual of how 
to do this work. Each district and school needs to work within its own context and use the 
framework to identify the critical elements to focus on for implementation. 

During this session, facilitators introduce handout 10.1 to guide dialogue. The discussion 
prompts can be revised to meet the needs of the district or school. If this workshop takes 
place over several sessions, there may have been internal dialogue about the kinds of guid- 
ance needed. Before or after the workshop, facilitators should consider input about who 
should be involved with this work over time. 



Using the optional assignment 


If participants have completed the optional assignment of developing a district or school case 
to review in topic 8, this is the time to bring it out for review. Areas of focus can be generated 
from the mapping process, by noting, for example, which areas have not yet received attention 
or where there have been barriers to practice. 



The majority of this session will be group dialogue and writing, and planning 
for next steps. Facilitators should check in with all groups during this stage. 
In the last 10 minutes of this segment, facilitators can bring the entire group 
together to debrief and clarify next steps. If facilitators are able to collect the 
work done in this session, it can be used to support deeper review and imple- 
mentation as next steps are taken. 
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Presentation slides and participant activities 


There is no roadmap 



The metaphor of the Iditarod dogsled race in slide 57 is apt for developing a culture of data 
use. In the original Iditarod, dogsled drivers had to get from point A to point B, but they 
did not have to follow a particular path. Rather than use precise route, drivers considered 
their expert knowledge of the terrain, the environment, and the conditions to make their 
way. The idea that there is no roadmap indicates that there is no step-by-step manual that 
can support a culture of data use. In developing a culture of data use, leaders require expert 
knowledge, a vision of what data use can look like in their setting, and a deep understand- 
ing of current practices in this domain. 



Other metaphors for not having a roadmap 


Share this metaphor, use your own, or ask participants to raise their own ideas about what it 
means to have no roadmap. 


The Iditarod metaphor does not imply that we have no idea how to create a culture of data 
use. Rather, it implies that each district will have to clarify its path forward by understand- 
ing and attending to the unique conditions in its setting. Knowledge of the terrain and 
existing conditions will help create a successful route, and attending to changing condi- 
tions over time will add to that success. 


Slide 58: 


Developing internal guidance to support a 
Culture of Data Use 


1 . Identify an area of focus for your work. Select one framework element 
in which to develop written guidance or policy. (Note, this is likely to be 
the same area you discussed in the last part of the agenda) 

2. If your district or school has developed a “case study” (optional 
assignment), this may help clarify priority areas for written guidance. 

3. Use Handout 1 0.2 in your workbook to explore the guiding questions 
in the one framework element you have identified. Consider and 
discuss ALL of the guiding questions in that element. 

4. Use the discussion prompts to have dialogue and craft an outline for 
written guidance in 1-2 areas. 

5. Clarify how you will work with your district or school leadership team to 
share and finalize the draft examples of data guidance documents that 
are developed in this session. 
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Show slide 58, which is a review of handout 10.1. These discussion prompts, in conjunction 
with handout 10.2, will lead to an initial draft of written guidance that can be shared with 
district or school leaders. For teams that might be having trouble identifying an area on 
which to focus, handout 10.3 provides some ideas of the types of policy that can be helpful. 
The next steps will likely focus on ensuring that the right district or school leaders have 
the opportunity to review it and that there is a plan to carry it through. 


Handout 10.2 

Guiding Questions/Culture of Data Use 


Slide 59: 


Participating in the flow of information 
for evidence-use 

Provide resources and assistance to 
make meaning from data 

Communicate professional 
expectations for data use 

Provide professional development on 
data use knowledge and skills 

Provide leadership to nurture a culture of data use 
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Ask participants to take out handout 10.2 (summarized in slide 59). Use this document 
to consider areas where guidance might be needed. At this point in the day, participants 
may be quite ready to narrow and focus on one or two areas where they see gaps and 
want to address that with written guidance. If that is the case, handout 10.2 might help 
them clarify and have some final dialogue about those one or two areas as they move into 
crafting written guidance. Following the brief review, give groups about 40 minutes to use 
handout 10.2. 


Slide 60: 


Developing guidance - debrief 



Report out: 

- What written guidance is 
being developed? 

- What barrier does this 
address? 

- In what way(s) does this build 
on current practice? 

- How will this guidance support 
more consistent 
implementation of a Culture of 
Data Use? 
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Bring the groups back together to report on their progress and next steps, as outlined in 
slide 60. At this time, it is helpful to capture and document agreed upon next steps and to 
collect notes and initial drafts of written guidance. Help clarify who has a primary role in 
moving this work forward and any timelines that have been identified. 
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Topic 11: Reflection and closing 


• Duration: 15 minutes 

• No handout 

• Slides 61-67 

Facilitator guidance 

This section allows participants to capture learning and supports group understanding of 
how that learning will help as participants develop a culture of data use in their setting. 
The slides focus on recalling the learning goals, success criteria, and guiding sessions for 
the workshop, providing some ideas to consider as they take the next steps, and then 
finally, sharing participant learning. 

Slides 61-65 are meant to be shared by the facilitator but not yet discussed. They lead 
up to an opportunity to discuss workshop outcomes on slide 66. Facilitators should move 
through slides 61-65 in about five minutes, leaving most of the session for presentation of 
content on slide 66. 

Presentation slides and participant activities 


Slide 61: 


Guiding Questions for the Workshop 


What does the research tell us about effective district and school 
data practices? What are the essential characteristics of successful 
data using cultures? 


What do successful data using cultures do? What are examples of 
the essential characteristics in practice? What can we learn from 
case studies of districts with successful data-using cultures? 

Which of these practices can be applied in your own setting to 
deepen/extend current practice? How can you create systemic 
change in data use practice using district or school-level guidance? 



■ AnTlDestEd • 



Ask participants to reflect individually on the degree to which these guiding questions 
were addressed in the workshop. 


Slide 62: 


Workshop Overview and Learning Goals 


Learning Goals: 

• Explore five characteristics that contribute to an effective culture of 
data use in schools and districts 

• Learn strategies to extend/deepen the culture of data use in your 
district and schools 


Success Criteria: 


Apply the five characteristics of the culture of data to your school or 
district practice - regardless of your stage of development 
Identify high leverage strategies you can use to strengthen the 
culture of data use in your school or district 


Identify policies or written guidance you can develop that will 
support more consistent implementation of data-use practices 
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Again, ask participants to pause and individually reflect on the degree to which these 
learning goals have been met. 


Parting thoughts 
Understand there is no roadmap 



Show slide 63 as a reminder that individual districts must clarify their own course, based 
on their current context and needs. 


Slide 64: 


Framing idea - a culture of data use 

In a data-using 


culture, daily use of 


evidence supports 


individual, team and 


organizational Jy 

learning that is 
reflected in new 
practices, norms and 
routines that further 
organizational goals. 
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Stress that developing a culture of data use is a core practice to help meet education goals. 


System integration 


Slide 65: 


District and school adoption of data-driven decision-making 
practices requires a focused coordination of elements of 
the larger education system, including such things as 
policy change, widespread professional development 
planning, and reliable and affordable support 
mechanisms. In other words, it is important to consider 
how the process will function across the entire district 
and will be used by participants at all levels of the 
system. (Means, Padilla and Gallagher, 2010) 
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Ask participants to read and reflect individually on this quote from researchers who offered 
support to this workshop. Hold discussion for the next slide. 
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Slide 66: 


Your ideas about a culture of data use 



As you think back to the vignette, 

case studies, tables and examples: 

• What will you do differently as a 
result of this session? 

• Regarding developing or 
enhancing a culture of data use, 
what have you learned or 
discussed today that motivates 
you to continue with this work? 

• What do you want to share with 
others about this session? 
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Use slide 66 to offer participants a chance to reflect on what they have learned. One strat- 
egy is to have them discuss these three questions in pairs or trios, then report on their 
discussions. 

• What will you do differently as a result of this session or workshop? 

• Regarding developing or enhancing a culture of data use, what have you learned 
or discussed today that motivates you to continue with this work? 

• What do you want to share with others about this session or workshop? 

Whatever facilitation strategy is used, it is important to hear participant reflections about 
their overall learning. Ask them to reflect back to examples of practice in the vignettes 
and case studies, and raise ideas around developing and implementing a vision of a culture 
of data use. 


Thank you 


Slide 67: 
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Handouts 


Pre-reading 


A practical framework for building a data-driven district or school: How a focus on data quality, 
capacity, and culture supports data-driven action to improve student outcomes . 

Authors: David Ronka, Robb Geier, and Malgorzata Marciniak 

© Public Consulting Group, Inc. Boston, MA. All rights reserved. 

Used with permission from Public Consulting Group. 

The current age of greater accountability in schools has challenged educators to seek effec- 
tive ways to incorporate data into their decisionmaking processes from the central office to 
the classroom. However, this is not just a matter of collecting more data. For data to inform 
decisions about policy, programs, practice, and student placement, three critical factors need 
to be taken into consideration: data quality, data capacity, and data culture. This White 
Paper describes a research-grounded model for data use and discusses these three factors, 
why they are important, and how they support effective data use in schools and districts. 

Schools rely on “random acts of improvement” (Bernhardt, 2006, p. 30) when educators 
do not set clear targets for improvement and then use data to track progress against mea- 
surable indicators to reach those targets. Data can be used to formulate appropriate and 
effective education policy and to measure the effectiveness of programs and instructional 
interventions. Data can also be used to measure individual student progress, guide the 
development of curriculum, determine appropriate allocation of resources, and report prog- 
ress to the community. But despite the leverage that can be gained by using data effective- 
ly, many schools still struggle with data-driven decision-making (Mason, 2002; Ingram, 
Louis, & Schroeder, 2004; Boudett & Steele, 2007; Stid, O’Neill, & Colby, 2009). This 
paper discusses a theory of action that links the conditions necessary for data use to the 
types of decisions that can be informed by data to improve student outcomes. The paper 
will present the overall theory of action followed by a discussion of the two primary com- 
ponents (conditions for data use and examples of data-driven decision making in schools) 
and will end with a discussion of the implications for district and school leaders. 

Theory of action 

Fifteen case studies published between 2002 and 2009 were analyzed to identify conditions 
in schools and districts that support data-driven decision making at the district, school, and 
classroom levels. Specific data-driven actions were documented within and across the cases 
in order to formulate a description of what effective data-driven decision-making looks like 
in a district and school. The theory of action that emerged is represented in the graphic 
below. It contains three foundational conditions for data use (conditions), that enable 
different types of data-driven actions related to policies, programs, practices, and student 
placement (actions), and that together are linked to improved student outcomes (results). 

According to the theory of action, if the necessary conditions for data use (data quality, 
data capacity, and data culture) are in place, and data are being used to formulate policy, 


© Public Consulting Group, Inc. Boston, MA. All rights reserved. Used with permission from Public 
Consulting Group. 
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Figure 1. Public Consulting Group’s data use theory of action 



Focused 

Results 
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Conditions 
for Data Use 



evaluate and design programs, guide practice, and place students in appropriate instruc- 
tional settings, then increased student achievement will result. However, it does appear 
that for data use to have a profound impact on student achievement, data use must be 
sustained over time, take place systemically throughout all levels of the organization, and 
be student centered. This theory of action, which emerged through a coding of the case 
studies, has been reinforced by our work with schools and districts over the past decade. 

Conditions for data use 

There has been much progress in the area of data use by educators at the district, school, 
and classroom level. However, many schools and districts still only use data superficially. 
Superficial data use happens when data are used inconsistently and/or inappropriately in 
pockets of the organization without systematic procedures, expectations, and accountability 
in place. In these environments, there may be some who engage in effective data use prac- 
tices. However, in the same school or district, data may also be used to punish educators, 
to justify the status quo, or to make critical placement decisions based on single data points 
(e.g., one assessment’s results) that restrict options and opportunities for students. Systemic 
data use, on the other hand, is where data are routinely and collaboratively used at all levels 
to inform organizational, program, and instructional improvement decisions directed at 
improving student outcomes. But this doesn’t just happen. It takes a concerted and deliber- 
ate effort for district and school administrators to put the necessary conditions in place that 
support and empower data-driven actions. In this paper we are primarily focused on student 
outcome data — that is, information about student learning (e.g., assessment or test data) 
and student engagement (e.g., attendance, conduct, graduation rates). There are many types 
of data that can inform schools of their progress toward goals (e.g., incidents of vandalism, 
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number of certified teachers, number of students enrolled in advanced classes). Our focus in 
this paper is on how schools and districts can most productively use data directly related to 
student outcomes to identify and understand issues related to curriculum, instruction, and 
assessment and make changes in how they operate in order to improve those outcomes. 

Successful conditions that were present in many of the case study schools and districts can 
be distilled into three categories: data quality, data capacity, and data culture. It appears 
that these conditions are fundamental to effective data-driven decision-making. These 
three areas synergistically interact to create an environment where data use is powered by 
high quality data, enabled by various data capacities, and supported by a culture of account- 
ability and collaboration. In the next sections of this paper, each of these is discussed. 

Data quality 

Access to high quality data can lead to greater levels of systemic data use and ultimately to 
improved student outcomes. 

Data quality includes: 

• Using multiple measures to ensure relevance and the ability to triangulate from 
more than one data set; 

• Making sure data are well organized and presented in data displays that are easy to 
interpret; 

• Using accurate data that have been standardized and cleansed; 

• Making data available to stakeholder groups before the data “ shelf life ’ has expired; and 

• Disaggregating data for analyzing across multiple factors. 

Without high quality data, stakeholder groups can lose faith in the value of data and 
become discouraged. At worst, educators can use poor quality data — data that are old, 
that are not disaggregated, or that are presented in confusing or inaccurate ways — and 
draw false conclusions about district or school needs. This can result in “data-driven” 
actions that can actually cause harm. It is important for districts and schools to put safe- 
guards in place to address data quality. 

Data capacity 

Data capacity is the next condition for data use. Without the capacity to access, under- 
stand, and use the data that are available, no amount of data (high quality or not) will lead 
to meaningful data use. In fact, without data capacity, the more data an organization has, 
the less it will be able to do with it. If data quality is the fuel, data capacity is the engine 
that converts the fuel to energy. Data capacity includes: 

• Organizational factors such as team structures, collaborative norms, and clearly 
defined roles and responsibilities that support data use; 

• Technology that can integrate data from multiple sources; 

• Data accessibility that allows multiple users to have access to data in formats that 
are easy to interpret; and 

• Data literacy and assessment literacy skills so data consumers know how to analyze 
multiple types of data and properly interpret results. 
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Schools and districts can improve data capacity by ensuring there has been adequate staff 
training on how to analyze and interpret test results, setting aside time for instructional 
and administrative teams to meet and discuss data, and establishing processes and proce- 
dures for accessing relevant data. 

Data culture 

A culture of data use can only develop if data quality and capacity are in place. A strong 
data culture results when an organization believes in continuous improvement and reg- 
ularly puts that belief into practice. Schools and districts that have a strong data culture 
emphasize collaboration as a keystone for success and they empower teachers and admin- 
istrators to make decisions for which they are held accountable. Elements of a strong data 
culture include: 

• Commitment from all stakeholder groups to make better use of data; 

• A clearly articulated vision for data use; 

• Beliefs about the efficacy of teaching and the value of data in improving teaching 
and learning; 

• Accountability for results coupled with empowering teachers to make instructional 
changes; 

• A culture of collaboration at all levels; 

• Modeling of data use by district and school leaders; and 

• Commitment to making ongoing instructional and programmatic improvements . 

Questions to consider when assessing the extent to which a culture of data use is present 
within a district or school include: 

• Is there commitment by all key stakeholders to use data for continuous 
improvement? 

• Are people held accountable for the use of data at the school and classroom level? 

• Is collaboration among staff highly valued? 

• Do school leaders model data-driven decision making as a key aspect of their roles 
and responsibilities? 

• Do teachers believe that data can and should be used to inform instruction? 

• Are teachers open to changing their instruction based on data about student 
learning? 

Data-driven action 

Data quality, capacity, and culture are the conditions necessary for systemic data use to 
exist within a school or district. But they are not the same as data-driven action. Rather, 
they are the foundation for data-driven action. Our analysis of the 15 case studies was 
framed by two key questions: What does a data-driven school or district look like? What 
kinds of data-driven actions do schools and districts take that successfully use data to 
improve student achievement? Four categories of data-driven actions emerged from our 
analysis. These categories also have been evident in our work with schools and districts 
across the United States and in Canada. Successful data-driven districts and schools use 
data in four key areas: to formulate sound policy, design and evaluate educational pro- 
grams, guide classroom practice, and inform student placement. 
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Policy 


Policy decisions lay the groundwork for educational practice. Data driven policies can have 
a powerful impact on needs assessment and planning processes, professional development, 
resource allocation, and teacher evaluation. Schools that model effective data use deter- 
mine overall school needs through data drawn from multiple sources. Student performance 
data are used to drive the school- and district-improvement planning process. Profession- 
al development is informed by gaps identified in student performance data as well as by 
instructional data collected during walkthroughs and classroom observation. Resources 
such as time and staff are allocated based on the identified needs of students, and student 
assessment data are used as supplementary information in the performance evaluation of 
teachers. 

Programs 

Educational programming is the vehicle for ensuring that instruction is appropriate, tar- 
geted to identified learning needs, and aligned to established curriculum frameworks and 
benchmarks. In schools and districts that strive to continuously improve student outcomes, 
data are used to identify best practices across classrooms, to identify gaps in the curriculum, 
and to determine which programs are effective and which programs should be discontinued. 

Practice 

What happens in school hallways and classrooms in terms of practice directly influences 
student learning. These are the habits and actions that, taken collectively, form a learning 
environment that either supports or hinders growth. Data-driven practices include sharing 
and discussing performance data with students and parents, using data to develop lesson 
objectives, and adjusting teaching strategies based on evidence of student learning. Exam- 
ples of what this looks like include teachers observing one another’s classrooms, leaders 
sharing data about progress toward school improvement goals, and instructional teams 
developing action plans to address specific areas of need identified through data analysis. 

Placement 

Finally, data should be used to ensure student placement into educational settings that are 
appropriate and optimally designed for student success. Teachers and administrators can 
use data to identify students who are at risk of academic failure or of dropping out, to guide 
flexible groupings of students for more focused and differentiated instruction, to identify 
appropriate supports and interventions, and to monitor the progress of students. 

What does effective data use look like in practice? 

In order to show how these conditions and data-driven actions look in actual schools 
and districts, this section of the paper presents five short descriptions of data use drawn 
from the 15 case studies that were analyzed. These “snapshots” reflect data use practic- 
es found in schools and districts throughout the United States during the past 10 years. 
These summaries demonstrate the interplay between data quality, capacity, and culture, 
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and demonstrate how data use practices emerge when leaders are deliberate about putting 
in place these conditions for effective data use, 

Supovitz and Klein (2003) conducted a study highlighting how different schools and dis- 
tricts use multiple measures to gauge student performance. They reported that the schools 
in their study drew achievement data from three primary sources: external standardized 
tests, schoolwide periodic formative assessments, and classroom-based customized assess- 
ments. The most prevalent of these sources was external data from the state and district. A 
few of the schools began to experiment with systematic school-wide assessments intended 
to provide interim feedback on progress toward school and grade-level goals. In classrooms, 
individual teachers fashioned creative and highly customized assessments. School leaders 
systematically analyzed a variety of student performance data at both the classroom and 
school levels. Rather than just relying on one individual test to provide guidance, innova- 
tive school leaders built more comprehensive systems of assessment that provided better 
interim information from multiple perspectives. By introducing this type of comprehen- 
sive system of assessments, teachers and school leaders could support an inquiry-oriented 
approach that involved ongoing and sustained investigations into the kinds of teaching 
that produced greater student learning. 

Assuring access to quality data turned out to be critical to reducing the dropout rate in 
one urban district (Stid, O’Neill, & Colby, 2009). The case study illustrated how a district 
with only 54 percent of its high school students graduating was able to significantly address 
the dropout problem over the course of one calendar year. The district collected data that 
allowed them to conduct an initial diagnostic analysis that focused on the characteristics 
of students who were dropping out of high school. On the basis of this analysis, middle 
schools prepared reports which listed struggling students and data about their academic 
performance, attendance, behavior, and information about whether they had faced certain 
life challenges (e.g., pregnancy and parenting, homelessness, placement in foster care). 
These reports were provided to high school leaders early enough in the school year for 
them to identify and implement focused and tailored interventions for these at-risk stu- 
dents at the beginning of their first year in high school. In the case of ninth-grade students 
from one high school, such actions based on the right data at the right time resulted in a 
25 -percentage-point reduction in the number of students experiencing three or more core 
class failures in the ninth grade, which was identified as a critical threshold to prevent 
students from dropping out. 

A study of six schools in another urban district (Mason, 2002) demonstrated the process 
of building capacity as a necessary intermediate step between collecting data and taking 
strategic action based on the data. The schools in the study faced several critical challeng- 
es: sustaining a commitment to transform data into knowledge, making data use a high 
priority, putting an effective data management and integration system in place, developing 
analytic skills in school leaders, and building capacity to link data to school improvement 
planning. The district engaged the schools in a two-year project that provided training 
and support. Some schools experienced moderate successes, but not without some hard 
lessons. Participants of the project realized how challenging it was to develop collaborative 
norms, build the necessary internal support for the data use initiative, build the capacity 
among staff to use and analyze data, and then apply that knowledge strategically. At the 
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end of the project, participants agreed that the process of using data needed a continuous 
and systematic focus, intensive professional development, and commitment to incorporate 
data use into everyday operations- 

Brunner et aL (2005) looked specifically at data use actions taken by effective teachers. 
The study reported that these teachers regularly used data to meet the needs of diverse 
learners, identify struggling students, create differentiated and individualized assignments, 
and provide learning materials appropriate to students’ levels. Teachers used data reports 
in conversations with other teachers, parents, administrators, and students. Many of the 
teachers used data to reflect upon the effectiveness of their own instruction and to shape 
their own professional development. Teachers also encouraged self-directed learning by 
giving the data to students to help them take ownership over their academic performance 
and learning. 

Ronka (2007) conducted interviews of school leaders at an elementary school during their 
first year of implementing a schoolwide data use initiative. The case demonstrates the 
importance of attending to the organizational and cultural aspects of introducing data use 
into the school environment. Specifically, the principal established a data team comprised 
of members who were representative of the school staff and who were critical to bringing 
about the kinds of programmatic and instructional change that might result from effective 
data use. The team met monthly throughout the year to monitor progress and to lay the 
groundwork for continuous data use by planning professional development on various uses 
of data, identifying data quality issues, taking action to address those issues, and coordi- 
nating data use across content areas and instructional teams. Stakeholder commitment at 
multiple levels was evidenced by the amount of time committed to planning and monitor- 
ing activities, and the principal’s strong leadership created an environment that was based 
on collaboration and focused on continual improvement. 

Implications for schools and districts 

In the case studies reviewed for this paper, each school or district applied a data-driven 
decision making approach for inquiry and action. The specific approach chosen, however, 
does not appear to be the major determinant for successful change over the long term. 
Making the approach “stick” requires a long-term vision for changing the way educators 
in the system make decisions and work to improve student results. It is this vision for 
changing the way decisions are made, when broadly communicated and shared throughout 
the organization, which guides sustainable growth through a particular data use approach. 
It is the task of district and school leaders to establish the vision and work toward it with 
strategic attention given to the three conditions for data use previously described. 

Using the theory of action presented in this paper as a guide, leaders can create strategic 
plans to improve data quality, capacity, and culture. This can lead to a productive inquiry 
and action process focused on improving the conditions that support effective data-driven 
action. The table below presents questions schools and districts can ask to identify areas 
for improvement in the three foundational conditions for data use. 
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Table 1. Conditions for data use: Guiding questions 


1 Conditions for data use 

Guiding questions I 

Quality 

• What data do we have that can help answer the questions we are currently 
asking about student learning? 

* What improvements to our data quality would expand our ability to ask and 
answer these and other questions? 

Capacity 

* What are the organizational structures and systems that enhance (or 
impede) our ability to use data effectively? 

Do all members of our school or district have the data they need to make 
effective decisions? 

Do all members of our school or district have the knowledge and skills 
necessary to make use of the data available to them? 

Culture 

* Are we basing the decisions we need to make on data and evidence? 
Are we using data to communicate our decisions in ways that foster 
engagement by all stakeholders in improvement efforts? 


Careful and thoughtful attention to the conditions in which data are being used is an 
essential component of leadership in today’s educational environment. The prolifera- 
tion of data and data systems has afforded educators the opportunity to fundamentally 
change the way they meet the needs of diverse students. When fostering and monitoring 
these conditions is a priority, then data-driven actions in areas related to policy, program- 
ming, practice, and student placement can be strategically focused on improving student 
achievement. 

Conclusions 

The theory of action presented in this paper advocates effective data use when making 
decisions about initiatives and instructional changes intended to improve student learning 
and achievement. When planning additions to the types and extent of data collection, 
enhancements to data systems, or data use professional development, we encourage edu- 
cation leaders at all levels to also consider the components of the theory presented in this 
paper. Assessing the extent to which specific strategic actions are supported by multiple 
types of data and a skilled culture of data use exists will enhance the likelihood that dis- 
trict and school improvement efforts will gain traction and ultimately lead to improved 
student results. 

Data use initiatives too frequently fail to thrive and grow because of inattention to one 
or more aspects of data quality, capacity, or culture. Initiatives to expand data collection, 
increase data access, or foster data use that are not connected to authentic and important 
data-driven actions (policy, programs, practice, and placement) are not sustainable over 
time if the extra work they require doesn’t lead to transformative change and positive 
student results. 
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Handout 1.1: Participant agenda and handouts overview 


1 Topic 

Learning objective 

Duration 

Slides 

Handouts | 

1: Welcome and 
background 

• Understand the role of Regional Education 
Laboratory Northeast & Islands and alliances in 
preparing this content. 

* Welcome and introduce facilitators and 
participants. 

5 minutes 

1-3 

1.1: Participant agenda 
and handouts overview 

2: Workshop overview and 
goals 

• Clarify what the workshop will cover and will not 
cover. 

• Review the agenda. 

• Identify overall learning outcomes and success 
criteria. 

15 minutes 

4-6 


3: Defining a culture of 
data use 

• Explore the idea that data literacy, by itself, is 
not sufficient to ensure data is used to guide 
instruction. 

• Clarify that a culture of data use implies that 
data-use practices happen when nobody is 
looking. 

20 minutes 

7-11 

3.1: Definition of a 
culture of data use 

4: Identifying participant 
and presenter assumptions 

• Uncover assumptions that individuals bring to 
this work. 

15 minutes 

12-13 


5: Introducing the Culture 
of Data Use Framework 

* Explain each of the five elements in the Culture 
of Data Use Framework. 

* Explore the research that undergirds the 
framework. 

* Understand that all five elements of the 
framework are equally important in practice. 

50 minutes 

14-27 

5.1: Culture of Data 
Use Framework: Initial 
reflections on five 
elements 

5.2: Culture of Data 
Use Framework: Five 
research summaries 

5.3: Culture of Data Use 
Framework: Findings 
from research 

6: Exploring district and 
school practices aligned 
with the Culture of Data 
Use Framework 

Develop images of practice for each element of 
the framework. 

* Begin to identify a range of practices that 
support work in each framework element. 

60 minutes 

28-34 

6.1: Culture of Data Use 
Framework: Vignettes 

6.2: Culture of Data Use 
Framework: Effective 
data-use practices 

7: Considering changing 
expectations of data use 

• Identify how education data-use practices have 
been framed in recent years. 

• Note current placement on a continuum of 
practice. 

20 minutes 

35-41 

7.1: Changing 
expectations of data 
use: Teacher activities 
on the continuum 

8: Exploring district data- 
use practices over time 

* Explore through a case study how district and 
school data-use practices have changed over 
time. 

Discuss a range of data-use practices that align 
to the framework. 

55 minutes 

42-45 

8.1: Culture of data use: 
District case study 
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Topic 


Learning objective 


Duration 


Slides Handouts 


9: Reviewing templates, 
tools, and strategies 
that support consistent 
implementation of data-use 
practices 

• Learn to use each of the framework elements 45 minutes 46-56 9.1: Discussion 

to guide discussions about current practice and protocol: Identifying 

next steps. implementation 

* Explore implementation barriers and strategies strategies for a culture 

to address those barriers. of data use 

9.2: Barriers to a 
culture of data use, by 
framework element 

9.3: Examples of policy 
and guidance to support 
a culture of data use, by 
framework element 

9.4: Examples of district 
and school guidance 
to support a culture of 
data use, by framework 
element 

10: Selecting strategies 
to further implement key 
framework elements 

Identify areas of focus for high-level strategies 50 minutes 57-60 10.1: Discussion 

to consider in local settings. prompts 

10.2: Guiding questions 
for developing policy or 
guidance to support a 
culture of data use, by 
framework element 

10.3: Examples of policy 
and guidance to support 
a culture of data use, by 
framework element 

11: Reflection and closing 

* Reflect on new learning. 15 minutes 61-67 

• Clarify next steps. 
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Handout 3.1: Definition of a culture of data use 


Definition of Culture of Data Use 


A strong data culture results when an organization believes 
in continuous improvement and regularly puts that belief into 
practice. Schools and districts that have strong data cultures 
emphasize collaboration as a keystone for success and they 
empower teachers and administrators to make decisions for 
which they will be held accountable. Elements of a strong 
data culture include commitment, vision, beliefs, 
accountability, collaboration, modeling, and commitment to 
ongoing instructional and programmatic improvements 

Ronka, Geier and Marciniak (2010) 


pi Regional Educational 
lUjjl- Laboratory at EDC 
& ISLANDS relnei.org 



Majtir Partners: 
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Handout 5.1: Culture of Data Use Framework: 
Initial reflections on five elements 


Ensuring access to data 

Data are accessed, coordinated, filtered, and 
prepared in ways that allow educators to quickly 
and efficiently analyze and interpret data to answer 
key questions and address important teaching and 
learning issues. 

Making meaning from data 

Protected time is provided to allow users of data to 
collectively make sense of what the data indicate 
and to explore how to move from data to evidence 
that will inform instruction. This time is focused on 
making meaning from data and is supported by the 
use of consistent inquiry-based practices. 

Clarifying expectations for data use 

There are clear expectations about how to use data, 
and these expectations change overtime as skills 
for data use grow. District, school, and teacher 
leaders frame consistent messages about how 
data can be used to support teachers’ professional 
capacity and student learning. 

Building knowledge and skills to use data 

Adequate professional learning takes place 
regarding data use, assessment literacy, and using 
data to inform instruction. Professional learning is 
integrated into daily practices and supports teachers 
in building content knowledge over time. Professional 
learning is differentiated to support teachers’ 
specific learning needs. 

Leading a culture of data use 

Leadership nurtures and supports a culture of data use and develops organizational structures that include 
time and resources to conduct ongoing data dialogue and feedback that will support users to act on new 
knowledge. Acting on knowledge is supported at the administrative, teacher, and student levels. Leaders’ 
use of data is central to helping educators interact around issues that will lead to improved learning 
outcomes. 
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Handout 5.2: Culture of Data Use Framework: Five research summaries 


Read the assigned summary and identify three to four key points to share with colleagues. 

Research summary 1: Ensure access to data: Participate in the flow of information for evidence use 

Investing in data management systems is an essential first step in supporting data-use prac- 
tices at the district and school levels (Honig, 2004; Means, Padilla & Gallagher, 2010; 
Wayman, Snodgrass-Rangel, Jimerson, & Cho, 2010). Decisionmakers at different levels of 
the education system have different information needs, and depending on their role, they 
need to access information in different ways. Developing an effective information system 
requires first identifying and analyzing the needs of decisionmakers (Breiter & Light, 2006; 
Park & Datnow, 2009). Any information system should be flexible enough to accommo- 
date multiple users of data (Mandinach, Rivas, Light, Heinze, & Honey, 2006) and should 
be structured to ensure that data are available in simplified and comprehensible forms. 
Ideally, data systems should be organized such that users can ask questions that address 
current education issues and needs (Means et al., 2010) and should allow teachers to focus 
on specific questions of student achievement (Hamilton et al., 2009). Marsh (2012) indicat- 
ed that a key element that increases the likelihood that data will be used for improvement 
is making data usable, safe, and easily digestible. 

In a national survey of data-use practices conducted by Means et al. (2010), teachers 
reported that they rarely prepared data on their own. Teachers relied on district, school, 
and data-team leaders to collect and prepare data for review. When lacking effective data 
management systems, teachers in one study reported significant time spent compiling data, 
resulting in their feeling constrained by a lack of time to explore instructional actions that 
might arise from data analysis (Wayman, Cho, Jimerson, & Spikes, 2012). Typically, the 
central office serves a critical role in organizing and providing data management systems, 
but schools have an important role as well. In a study of urban school leadership related 
to data use (Knapp et al., 2010), principals in many schools used the district’s data system 
only as a starting point. In these schools, principals and teacher leaders created their own 
within-school data systems to provide continuous feedback to teachers and teacher leaders 
about student learning. This frame of having both district and school data systems relates 
to the ideal of having differentiated access based on the needs of data users. 

As leads in ensuring access to data, districts are also cautioned not to be singularly direc- 
tive in their role as data managers. In a study of central office processes related to district 
data use, Honig and Venkateswaran (2012) found that the role of the central office is to 
support data use both u top down” and “bottom up.” Central offices that encouraged strong 
bottom-up flow of information had stronger evidence-use practices and were more likely to 
access important information from schools that would inform central office support over 
time. Viewed from a different vantage point, in a study of social networking, Daly and Fin- 
nigan (2010) reported that when schools have weak ties with the central office, the “top 
down” nature of information from the central office can limit rather than enable schools’ 
use of data in decisionmaking. Ensuring access to data that are useful and transparent 
seems to be helped by ensuring strong central office and school communication about the 
purposes and applications of data use at the school level. 
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Summary of research findings 


Themes : Access , usability , transparency , ease of use: 

• Districts work in cooperation with schools to develop data systems that ensure 
appropriate data for classroom, school, and district use, 

• Districts and schools coordinate how to centralize and streamline data reporting. 

• Districts and schools work together to clarify when data analysis needs are chang- 
ing and to revise systems to accommodate emerging needs. 

• Both the central office and schools may have a role to ensure that data reports 
meet the needs of teachers and can address the questions of teacher teams. 

Research summary 2: Clarify expectations: Communicate professional expectations for data use 

Hamilton et al. (2009) recommends establishing a clear vision for data use that includes 
a written plan that articulates activities, roles, and responsibilities for all data users in the 
system. Developing a common vision is an entry point toward consistent communica- 
tion regarding expectations for data use. Common understandings provide a vehicle for 
articulating goals and for fostering meaningful conversations about teaching and learning 
(Wayman, Jimerson, & Cho, 2012). At the school level, expectations for data use are com- 
municated primarily by principals and teacher leaders and can be conveyed in multiple 
ways. Schedule changes that promote time for teacher reflection and dialogue, for instance, 
help communicate that data-driven inquiry has value and will be supported through struc- 
tured time. Similarly, schools that document common understandings that emerge from 
team dialogue can move closer to an aligned vision of teaching and learning (Wayman, 
Snodgrass-Rangel, Jimerson, & Cho, 2010). 

Spillane (2012) describes how schools redesign organizational routines and expectations 
to continually frame the changing expectations regarding instructional practices. In 
these environments, over time, data can help to establish more consistent language and 
instructional practice throughout the school (Wohlstetter, Datnow, & Park, 2008). Setting 
expectations at the district level is an important role of central office leaders. Establishing 
consistent data-use practices that are used at the principal and school levels appears to 
support more focused data-use practices (Park & Datnow, 2009; Honig & Venkateswaran, 
2012). Central office administrators also play an informal role in disseminating under- 
standings of data-use practices. The degree to which district leaders have positive informal 
relational networks among school staff supports data use in schools (Daly, 2012). Marsh 
(2012) suggests also that relational trust within schools is an important prerequisite to data 
use and that the ways in which principals and teacher leaders communicate clear expec- 
tations about the use of protocols, norms, and language can impact teachers’ comfort with 
data-use practices. 

Principals’ ability to filter information from the central office, to shift from messages about 
accountability to messages focused on the central issues of teaching and learning, may 
be able to strengthen the impact of key messages at the school level (Knapp, Copland, 
Honig, Plecki, & Portin, 2010). School leaders’ ability to reframe district messages helped 
teachers internalize improvement practices. In other words, principals reported that when 
leaders found ways to capture district messages to help support schools’ own improvement 
practices, the improved coordination helped support internal values of teachers working 
collaboratively to meet internal school improvement goals. 
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Summary of research findings 


Themes: Vision, alignment , consistency, networks: 

• Districts and schools establish and communicate a common interpretation and 
orientation toward data-driven decisionmaking. 

• The district and schools provide clear messages about how data use supports 
improvements in student learning. 

• District data-use expectations are mediated at the school level by formal and infor- 
mal school leaders, so that establishing professional expectations shifts over time, 
as the disposition and skills to use data grow. 

• District and school leaders clarify when data needs are changing. 

• Expectations for data use are communicated through presentations, policy docu- 
ments, and modeling of expected practice. 

Research summary 3: Provide resources and assistance to make meaning from data 

Research has fairly well established that analyzing data, and moving from analysis to 
instructional action, is most likely to occur in collaborative teams. Data use in schools 
is primarily a process of interpretation (Coburn, 2010; Lachat & Smith, 2005) in which 
teachers are engaged in making sense of data (Spillane, 2012; Little, 2012; Wohlstetter 
et al., 2008). Collaborative teams provide a vehicle for this analysis. Structured collabo- 
rative time for teachers is essential to help teachers move from analysis of data to instruc- 
tional action (Wayman & Stringfield, 2006). Collaborative team practices should take 
place regularly within subject area and grade-level teams (Hamilton et al., 2009), where 
data analysis can be focused on developing common expectations for student learning and 
consistent instructional practices (Lachat & Smith, 2005; Halverson, Grigg, Pritchett, & 
Thomas, 2007). Teams can also be structured as vertical or cross-subject- area collabora- 
tion, to provide time for teachers to develop and align consistent instructional practices 
and review broader student-learning needs (Knapp, Swinnerton, Copland, & Monpas-Hu- 
ber, 2006; Datnow, Park, & Wohlstetter, 2007). Marsh (2012) highlights the value of both 
horizontal- and vertical-teaming practices as an important component of successful inter- 
ventions. Similar findings are described in a case review of data-use practices in high-per- 
forming, high-poverty schools (Gleason & Gerzon, 2013), where vertical teams deepened 
teachers’ content knowledge and horizontal (grade-level or departmental) teams focused 
on developing teachers’ use of inquiry. 

Using an inquiry approach to data analysis in collaborative teams is a fairly well estab- 
lished practice (Nelson, Slavitt, & Deuell, 2012). A variety of inquiry models are reviewed 
in the literature (see Hamilton et al., 2009; Mandinach, Honey, Light, & Brunner, 2008; 
Means et al., 2010; National Forum on Education Statistics, 2012; Ikemoto & Marsh, 
2007), and emerging evidence suggests that teams’ use of a structured cycle of inquiry can 
lead to improvements in student learning. Gallimore, Ermeling, Saunders, and Goldenberg 
(2009) found that more frequent data team meetings led to improved practices. Similarly, 
Slavit, Nelson, and Deuel (2013) frame that the use of an inquiry cycle can support teacher 
learning that is ongoing and focused on instruction, teachers’ knowledge of content, and 
learning goals. 

Collaborative teams appear to benefit from a sense of safety (Marsh, 2012; Means et al., 
2010) that ensures that conversations about data, and the data themselves, will not be used 
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in punitive ways- Teams also benefit from relational trust within the group (McLaughlin & 
Talbert, 2006; Talbert, 2009), the use of well established norms (Louis, 2006), and having 
at least one person on the team with strong content knowledge (Nelson, Slavit, & Deuell, 
2012)- Collaborative teams also benefit from a distributed leadership model, whereby team 
members are provided with opportunities for leadership and ongoing professional learning 
(Hargreaves & Fink, 2006). 

For collaborative inquiry to support data analysis that leads to instructional (and student 
learning) changes, teachers must be willing to work in teams to explore their current 
instructional practices in light of evidence (Little, 2012; Horn & Little, 2010)- Nelson et ak 
(2012) explore the idea of measuring the conversational routines of teams- In a five-year 
analysis they determined that professional learning communities with conversational rou- 
tines that are consistently focused on improving student learning resulted in more trans- 
formational changes in teachers’ beliefs, values, and instruction compared with teachers 
whose conversational routines focused on proving what students knew- Groups focused 
on improving student learning through inquiry carefully examined student data, openly 
wondered about what they could do differently to better support student learning, and 
were willing to change their practice- In these teams “knowledge became a dynamic, 
ongoing negotiation of learning goals, student understandings, and implications on prac- 
tice” (Nelson, Slavit, & Deuel, 2012, p- 16). Further, the authors submit u a teacher group’s 
stance toward student learning data can determine the nature of their collaborative work” 
(Slavit et ak, 2013, p. 1). 

Summary of research findings 

Themes : Inquiry , trust, norms, safety, relationships: 

• District and school staff work together to ensure that teachers have adequate 
structures and supports to review data. 

• Educators are supported in participating in collaborative inquiry in order to make 
sense of data and apply findings to instruction and improvement. 

• Educators from multiple levels of the education system work together to collective- 
ly understand how to use evidence from data analysis in decisionmaking. 

• District and school leaders work together to ensure that teachers apply new knowl- 
edge to improve classroom instruction or school-level practices. 

Research summary 4: Provide professional development on data-use knowledge and skills 

In their study of current data-use practices in U.S. schools, Means, Padilla, DeBarger, and 
Bakia (2009) identified that teachers report two major barriers to implementing data use: 
a lack of preparation on how to use data and a lack of staff technical skills to use data 
systems- Teachers’ level of individual skills and knowledge is essential to being able to 
make meaning from data in collaborative teams (Marsh, 2012). Professional development 
for data use helps teachers develop skills for collaborative work, such as understanding 
data literacy and assessment literacy (Mandinach & Gummer, 2013), applying interpretive 
frames of reference for data analysis (Knapp et ak, 2006), and understanding how to move 
from data analysis to using information to support instructional or administrative practice 
(Mandinach & Honey, 2008; Mandinach & Gummer, 2013). Means et ak (2010) note that 
only half of districts surveyed provided training to teachers on how to use data to inform 
instructional practice. 
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Professional development that focuses on educators’ expanding their repertoire of instruc- 
tional strategies helps them better respond to the needs of individual students or groups of 
students identified during data analysis (Breiter & Light, 2006), This focus on deepening 
instructional practice through data, with regards to professional development, indicates 
that professional learning should be customized to meet the specific needs of teachers. The 
Means et aL (2010) survey indicates that a majority of teachers want more training on how 
to interpret data and connect it to instructional practices. Orland (2012) suggests that data 
literacy for teachers must necessarily differ by content area because the kind of data that 
teachers are asked to interpret differs from grade to grade and across curriculum areas. In 
other words, professional development should be differentiated based on teachers’ needs 
(Wayman, Snodgrass-Rangel, Jimerson, & Cho, 2010) and focused on deepening content 
knowledge to help them identify instructional changes that will lead to student improve- 
ment (Timperley, 2009). 

Honig, Copland, Rainey, Lorton, and Newton (2010) identify the role of the central office 
as essential to supporting professional development for data use in schools. Central office 
leaders appear to be a main provider of professional development and have a particular 
focus on helping school staff build capacity to use evidence. Datnow, Park, and Wohl- 
stetter (2007) recommend providing professional development for district leaders who can 
then provide turnkey training to district and school staff as necessary. This is a strategy for 
building internal capacity and for developing consistent practices across schools. Wayman, 
Snodgras-Rangel, Jimerson, and Cho (2010) note that when a district did not embed data 
use and associated learning opportunities in the regular workday, teachers opted out of 
data-use practices entirely because of the additional work they entailed. 

Summary of research findings 

Themes: Content knowledge , data literacy , assessment literacy, internal capacity building: 

• Districts and schools provide opportunities for professional learning that builds 
educators’ capacities to identify data, interpret data, make meaning from evidence, 
and use evidence to inform instruction. 

• Professional development should combine information about data literacy and 
assessment literacy with content expertise to build knowledge of how to apply data 
findings. 

• Learning opportunities should include expanding teachers’ repertoire of instruc- 
tional strategies to ensure that teachers can more effectively transition from analy- 
sis to classroom practices that are informed by the evidence. 

• Whenever possible, learning opportunities should take place during the school 
day and be conducted by internal leaders. 

Research summary 5: Provide leadership to nurture a culture of data use 

Leaders’ recognition that data analysis processes support the real work of teaching and 
learning appears to serve as an important leadership frame through which data analy- 
sis can take hold. In this way, leading for data use is largely consistent with leading for 
improvement (Knapp et aL, 2010). In a study of urban leadership practices, Knapp and 
his colleague find that leadership for data use is strongly focused on issues of teaching and 
learning, identifying innovative resources to support teachers and teacher leaders, structur- 
ing time to attend to issues of teaching and learning, and providing a consistent focus on 
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using evidence to guide instructional improvement- Schools with a high level of data use 
exhibit more effective practices to support teachers to use data, including providing struc- 
tural supports to ensure time for collaborative team meetings (Way man, Cho, Jimerson, 
& Spikes, 2012), modeling data-use practices with faculty (Young, 2006), and distributing 
internal expertise among faculty (Anderson, Leithwood, & Strauss, 2010). 

Schools with a focus on continuous improvements in learning often see a “proliferation 
of individuals engaged in within-school instructional leadership” (Knapp et aL, 2010, p. 
11)- Involving multiple teacher leaders in playing roles in supporting collaborative inquiry, 
professional development, and communication allows for more internal capacity building 
related to developing inquiry-based practices. Deepening the connection to student learn- 
ing can be enhanced through the use of distributed leadership models. Gallimore et aL 
(2009) found that using teacher-facilitators during inquiry team meetings opened the door 
for coaches, content experts, and principals to take on more informal leadership roles in 
meetings, providing necessary support and leadership to teacher teams. In this way, infor- 
mal teacher leaders provide much needed support to ensure that data practices can take 
hold. Similarly, when principals support teachers and teacher leaders in sharing developing 
strategies for data analysis, it can lead to more effective team practices, as team members 
come to understand and implement more useful collaborative inquiry practices (Nelson, 
Slavitt & Deuell, 2012). 

Central office personnel play an important leadership role in developing a culture of data 
use. As leaders, they can frame key messages, provide resources and supports for school 
implementation, convene cross-school dialogue groups to streamline data-use practices, 
and develop strategies to build capacity for data use over time. A central focus of their 
role is to support principals to use data themselves. Luo (2008) finds that if the central 
office implements models in which principals are held accountable to use data, there is 
an overall positive impact on principals’ use and comfort with data. The central office 
can help schools set aside time needed for collaborative inquiry and streamline the use of 
inquiry practices (Wayman, Cho, Jimerson, & Spikes, 2012). 

Leadership for data use is not a one-way street, nor is it only top-down. Data-use processes 
in central offices depend on schools. Honig and Venkateswaran (2012) note that school 
staff can support central office staff to make sense of evidence about school progress and 
can help the central office use this evidence in its decisionmaking. This two-way focus 
toward leading for data use may ensure greater transparency in data use. Marsh (2012) 
identified data transparency as a key element that increased the likelihood that data would 
be used for improvement, because transparency assures teachers that they will not be eval- 
uated based on data findings. 

Summary of research findings 

Themes: Distributed leadership , shared roles, differentiated support, teaching and learning 

• The principal recognizes and models how data use informs instruction and fosters 
shared mental models of how data use can improve teaching and learning. 

• Principals and superintendents ensure access to resources that establish a data 
culture, such as setting aside time for data practices, ensuring a safe environment 
for teachers to engage in dialogue about best practices, and modeling effective 
data-use practices. 
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• The role of the central office includes ensuring that the principals’ data responsi- 
bilities are clearly defined and manageable. 

• The central office should both empower principals to use data to make decisions 
and hold principals accountable for data-use practices in their schools. 

• Leaders report having more successful data-use and inquiry practices using shared 
or distributed leadership models that include teacher leaders in a variety of roles. 
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Handout 5.3: Culture of Data Use Framework: Findings from research 


Provide resources and assistance to make 
meaning from data 

District and school staff work together to ensure 
that teachers have adequate structures and 
supports to review data. 

• Educators are supported in participating in 
collaborative inquiry in order to make sense 
of data and apply findings to instruction and 
improvement. 

Educators from multiple levels of the education 
system work together to collectively understand 
how to use evidence from data analysis in 
decisionmaking. 

District and school leaders work together to 
ensure that teachers apply new knowledge to 
improve classroom instruction or school-level 
practices. 

Provide professional development on data- 

use knowledge and skills 

Districts and schools provide opportunities for 
professional learning that builds educators’ 
capacities to identify data, interpret data, make 
meaning from evidence, and use evidence to 
inform instruction. 

• Professional development should combine 
information about data literacy and assessment 
literacy with content expertise to build knowledge 
of how to apply data findings. 

* Learning opportunities include expanding 
teachers’ repertoire of instructional strategies 
to ensure that teachers can more effectively 
transition from analysis to classroom practices 
that are informed by data. 

* Whenever possible, learning opportunities 
should take place during the school day and be 
conducted by internal leaders. 

Provide leadership to nurture a culture of data use 

• The principal recognizes and models how data use informs instruction and fosters shared mental models 
of how data use can improve teaching and learning. 

• Principals and superintendents ensure access to resources that establish a data culture, such as setting 
aside time for data practices, ensuring a safe environment for teachers to engage in dialogue about best 
practices, and modeling effective data-use practices. 

• The role of the central office includes ensuring that the principals’ data responsibilities are clearly defined 
and manageable. 

• The central office should both empower principals to use data to make decisions and hold principals 
accountable for data-use practices in their schools. 

• Leaders report having more successful data-use and inquiry practices using shared or distributed 
leadership models that include teacher leaders in a variety of roles. 

Source: Hamilton, Halverson, Jackson, Mandinach, Supovitz, & Wayman, 2009; Honig & Venkateswaran, 
2012; Mandinach and Jackson, 2012; Means, Padilla, and Gallagher, 2010; Spillane, 2012; Wayman & 
Conoly, 2006; Wayman, Jimerson, & Cho, 2012; Wayman, Jimerson, & Cho, 2010. 


Participate in the flow of information for 
evidence use 

Districts work in cooperation with schools to 
develop data systems that ensure appropriate 
data for classroom, school, and district use. 

• Districts and schools coordinate how to centralize 
and streamline data reporting. 

Districts and schools work together to clarify when 
data analysis needs are changing and to revise 
systems to accommodate emerging needs. 

• Both the central office and schools may have a 
role to ensure that data reports meet the needs 
of teachers and can address the questions of 
teacher teams. 


Communicate professional expectations for 
data use 

Districts and schools establish and communicate 
a common interpretation and orientation toward 
data-driven decision making. 

• The district and schools provide clear messages 
about how data use supports improvements in 
student learning. 

District data-use expectations are mediated at 
the school level by formal and informal school 
leaders, so that establishing professional 
expectations shifts overtime, as the disposition 
and skills to use data grow. 

District and school leaders clarify when data 
needs are changing. 

* Expectations for data use are communicated 
through presentations, policy documents, and 
modeling of expected practice. 
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Handout 6.1: Culture of Data Use Framework: Vignettes 


Element I: Participate in the flow of information for evidence use 

This rural high-poverty K-12 district has a limited data infrastructure. It has recently 
implemented an off-the-shelf student information system (SIS), which is poorly understood 
and has had numerous technical glitches. The person tasked with the role of technology 
director teaches technology at the local high school and, while knowledgeable about soft- 
ware, has limited knowledge of infrastructure. There are no data coaches and one K-8 
literacy coach who travels to three schools. The district has extremely limited financial 
resources. 

When a new district superintendent arrives, she recognizes that data-use practices are 
extremely limited at both the district office and at the classroom level. The superintendent 
establishes a district data leadership team, whose role is to prepare data for review, develop 
visual data displays, and serve as the “lead learners” who will learn about and disseminate 
information on key data-use practices. The district data leadership team membership is: 

• District technology coordinator. 

• District curriculum coordinator. 

• District K-8 literacy coach. 

• Two lead teachers from each school. 

• Two principals. 

• Two guidance counselors. 

In its first year, the team focused on learning about data-use practices in schools. It attend- 
ed two summer trainings run by the state department of education, one on using data in 
schools (with Nancy Love, author of The Data Coaches Guide ) and the other on data-based 
dialogue (with Bruce Wellman and Laura Lipton, authors of Data Driven Dialogue: A Facil- 
itators Guide to Collaborative Inquiry ). At the urging of the superintendent, it planned to 
“get the lay of the land” prior to implementing any new data initiatives. 

In its first year, the team met monthly to complete the following tasks: 

• Met with all teachers (by school-based, grade-level teams) in the district to discuss 
current data-use practices. 

• Developed an assessment map documenting the current assessments in use at each 
grade level and in each school. 

• Developed an assessment inventory documenting available data for program and 
curricular review. 

• Researched how high-performing schools are using data with teachers and at the 
central office. 

• Researched data-management systems to replace or complement the current 
student information system. 

• Learned how to analyze commonly used data types, in preparation for teaching 
teachers how to streamline analysis. 

• Analyzed issues and prepared a report related to data collection and data quality 
in the district (and its relationship to the quality of data the district is reporting to 
the state). 

• Created teacher focus groups to give twice-yearly feedback to the district data 
leadership team. 
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• Prepared data reports showing critical information on state assessment results. 

• Made recommendations to the superintendent and school board related to new 
technology and staffing to support data-based practices (including a new student 
information system and a data-entry clerk position). 

• Worked with school leaders to establish data-use role expectations for the princi- 
pals and sent principals to five-day summer intensive training on leading data use. 

In the second year, the District Data Leadership Team met biweekly to complete the fol- 
lowing tasks: 

• Revised the assessment map to develop consistent common assessments (across 
schools) by grade level and subject area. 

• Created a plan to introduce new student assessment tools (beginning the follow- 
ing year) to address gaps in assessment. 

• Developed new job descriptions for the technology director and the curriculum 
director, to focus on two discrete aspects of data systems and use. 

• Established new written guidance on the principal’s role in data use and incorpo- 
rated this into the principal’s job description. 

• Provided guidance and direction to the newly hired data entry clerk. 

• Continued twice-yearly focus groups with teachers at each school related to data 
use and professional development. 

• Developed a partnership with the local university to run an assessment literacy 
course for interested teachers, at reduced cost and available for graduate credits. 

• Worked with principals to identify lead teachers who will receive intensive profes- 
sional learning to lead data teams. 

• Continued to advocate for a data warehouse system and to create written justi- 
fication for the expense to present to the town fiscal committee during budget 
negotiations. 

Element II: Communicate professional expectations about data use 

This suburban school district introduced several new district assessments within a year of 
being identified as “in need of improvement.” In addition to district-mandated assessments, 
schools were also adding new assessments where they recognized gaps. It was a confusing 
time, as neither school leaders nor teachers were clear as to how these assessments would 
be analyzed or by whom. Different leaders in the district expressed different expectations, 
with some leaning on district or principal analysis of new benchmark data and others indi- 
cating teachers should be the primary users of all student assessment data. 

The district leadership team decided to take action by developing a one-page overview doc- 
ument to clarify expectations about different assessments, identify who was to use different 
types of assessment data, and begin to clarify training and support needed for different 
users of different types of data. Table 6.1 outlines its initial work, which was co-developed 
by district and building leaders. 

Over the course of the year following the development of table 6.1, the district curriculum 
director met with school principals monthly to clarify how data and assessment work was 
progressing, to identify areas for professional development, and to guide the work of district 
academic coaches. During these meetings principals were expected to bring examples of 
data, to work together to learn how to review new forms of data, and to clarify expectations 
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for teacher and administrator use of various types of data. The curriculum director worked 
with principals to develop buildingwide strategies to model data use and to use data to help 
clarify and identify areas of instructional need and program review. 

Work over the year led to a more detailed policy document, created by the district and 
then refined at the schools, which further clarified roles and responsibilities for data use. 

Element III: Provide resources and assistance to make meaning from data 

This Title I suburban elementary school has been focused on increasing data-use practices 
for several years. Its focus has been on developing a building-based structure that allows for 
teachers to use data to inform school improvement — every day — through an integrated 
team structure. Each teacher participates in a weekly grade-level data team meeting, which 
reviews current academic data in literacy and math and plans for next steps in instruction. 
Teachers are members of two additional teams that each meet biweekly. One is a vertical 
team focused on developing academic content through the grades, and the other is a learn- 
ing team focused on researching and introducing new pedagogy that teachers identify each 
year, such as technology, formative assessment, and supporting a diverse student body. 

These teams intersect around using data to inform school improvement. Each team 
meeting begins with a review of relevant evidence — the data could be academic, demo- 
graphic, behavioral, or program-based. Every team is responsible for using data to deter- 
mine an understanding of baseline practice and for setting measurable performance goals 
for teamwork. In some cases these performance goals are for student learning; in other 
cases they are for program implementation or teacher adoption of new practices. 

The role of grade-level data teams 

Academic coaches as a resource . Grade-level data teams focus on review of commercial 
diagnostic assessments (Dynamic Indicators of Basic Early Literacy Skills, Developmen- 
tal Reading Assessment) as well as monthly district benchmark assessment data. During 
the first five years of operation, two academic coaches (one in math and one in literacy) 
received additional training to support the use of tools and protocols in data analysis. Prior 
to data team meetings, academic coaches would prepare data displays for team review, 
select data-use protocols for analysis, and create spreadsheets that facilitated the analysis 
of multiple data points. Academic coaches also facilitated placement in the response to 
interventions, based on current results. The primary focus of teachers during data team 
meetings is to explore instructional actions to support next steps in student learning. 

Peers as a resource . A unique aspect of the school’s data team practice is a structured model 
to encourage teachers to learn from other grade-level teachers “in the moment.” At any 
time, if a teacher is not certain how to implement a new instructional technique raised in 
the team meeting, he or she can visit a peer within a day or two to watch the instructional 
practice under discussion. The visiting teacher has his or her class covered and visits (with 
the academic coach) to observe a lesson or two. These structured reviews, supported by 
academic coaches, help teachers “make sense” of how to revise instruction based on data 
and over time create agreement among grade-level team members about what constitutes 
“rigor” and what it means to “raise the bar” for all students. This peer visitation model is 
used frequently by many veteran and novice teachers. 
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Table 6.1. District academic assessments: Determine types and roles for data analysis 


Assessment 


Expectations for users of data 

Focus for professional 

type(s) 

Assessment tools 

Teachers 

Students 

School/district 

learning 

Daily: Classroom 
formative 

Questioning 
White boards 
Exit tickets 

Observational 

records 

Monitor student 
understanding of 
learning goals and 
revise instruction 
based on data 

Self-monitor 
progress toward 
learning goals and 
make adjustments 
as needed 


Use data to set learning 
goals with students 

Use data “on the fly” to 
change instruction 

Develop flexible groups 
Track formative data 

Periodic: 
Diagnostic 
assessments 
and progress 
monitoring tools 

Dynamic Indicators 
of Basic Early 
Literacy Skills 

Developmental 
Reading Assessment 

Aimsweb (math) 

Place students 
into intervention 
groups, using the 
district response to 
intervention mode 

Increase 

understanding of 

student-learning 

needs 

Self-monitor 
progress toward 
learning goals 

Understand 
placement decisions 

Provide early 
identification of 
students who 
require targeted 
interventions 

Diagnose individual 
knowledge and skills 

Group students for 
instruction 

Establish intervention 
groups 

Identify next steps for 
learning 

Weekly or 
biweekly: 
“Common” 
assessments 
designed by grade 
level and aligned 
to standards and 
calendar 

Weekly assessments 
developed by teacher 
teams 

Evidence reviewed 
in grade-level data 
teams to guide 
instruction 

Check individual 
and class progress 
against learning 
goals 

Guide dialogue 
about next steps in 
instruction 

Check progress 
toward learning goals 

Clarify what has 
been learned and 
what comes next 

Identify students 
who require 
additional supports 
and students who 
are ready to move to 
higher level work 

Identify patterns in 
proficiency 

Placement considerations 

Guide re-teach and enrich 
groups 

Periodic: 

Unit testing: 
high school 
competency 
assessments 

End of unit tests 

High school 
competency 
assessments 

Determine grades 
and promotion 

Gauge progress 
toward meeting 
standards and share 
with parents at 
conferences 

Conduct department- 
and grade-level team 
analysis of mastery 

Clarify grading and 
reporting for formative 
versus summative 
assessments 

Communicate with parents 

Quarterly: 

Benchmark 

assessments 

Fountas and 
Pinnel Benchmark 
Assessment 

District benchmark 
assessments in 
math, science, and 
social studies 

Northwest Evaluation 
Association 
assessments 
(grades 6-12) 

Analyze which 
students need 
additional 
interventions or 
targeted services 
and which can 
benefit from 
advanced work 

Analyze learning 
trends for individual 
and groups of 
students compared 
to national norms 

Gauge progress 
toward meeting 
standards and share 
with parents at 
conferences 

Analyze trends in 
student performance 
overall and across 
grades and buildings 

Identify which 
students are meeting 
the standards and 
which are not 

Identify area of student 
growth 

Use effective instructional 
strategies 

Identify patterns in 
proficiency 

Placement considerations 

Annual: State 
assessment 

New England 
Common 

Assessment Program 

Analyze curricular 
focus areas for 
improvement 

Align curriculum to 
state standards 


Identify broad 
areas of strengths 
and weakness in 
the curriculum, 
determine degree 
of progress, and in 
which areas students 
are not meeting 
standards 

Train leadership to use 
common protocols at the 
school to support effective 
use of state assessment 
evidence, including 
patterns of student 
achievement, instructional 
programs, additional 
knowledge, skills, or 
resources for staff 
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Immediate system response to assessment results . As teachers are given multiple supports to 
meet the needs of students, they are also held accountable to maintain those high standards. 
If, for example, benchmark data show that a certain percentage of students are not meeting 
performance targets, coaches respond immediately. Coaches work with teachers to figure out, 
and resolve, any outstanding issues. This might mean that a more expert coach takes over 
the teaching role for several days, while the classroom teacher observes, to showcase issues of 
content delivery, scaffolding, pacing, formative assessment, or grouping strategies that address 
identified learning needs. Teachers report that this process pushes them to pay attention to 
daily instructional data to ensure that students are on track to master standards. 

The role of vertical teams 

Vertical teams are led by teachers, and their main goal is to set performance goals, outline 
agreed-upon learning outcomes for key standards, and align assessments to those stan- 
dards. As content is revised, vertical teams set a structure for ongoing review of academ- 
ic content to consider alignment among grades and to look at student development over 
time. Teams review data from benchmarks across grades, review classroom grading over 
time on key academic standards, and identify trouble spots in the curriculum that may 
need more resources or professional learning. Through alignment tools, the vertical teams 
ensure that the intended curriculum is taught and that any content-based training is fully 
implemented in the classroom (and if not, they identify the supports necessary to ensure 
full implementation). Team members work with their grade-level teams to share findings 
from vertical meetings and, in particular, to review assessment data across grades. Time is 
set aside at grade-level meetings for vertical team representatives to review data-analysis 
findings in each content area and to adjust instruction accordingly. 

The role of learning teams 

Learning teams are organized by teachers and designed to develop areas of interest among 
faculty. Teachers self-select from a variety of learning teams, each focused on using an action 
research model to bring new instructional resources and tools to the full faculty. Learning teams 
pilot new techniques, and if these techniques are successful in engaging students or improving 
outcomes, a new instructional practice or program is brought to the faculty for review. 

Learning team meetings include review of a wide variety of types of data — including 
demographic, program, behavioral, and climate data. Thus, teachers need to know how 
to look at various types of data displays and must be knowledgeable about the inferences 
that can be made given specific datasets. Over time, the principals and academic coaches 
work with each learning team to develop skill sets in data literacy — conducting mini-les- 
sons on such topics as how to read different tables, graphs, and data displays, determining 
what inferences can be drawn from certain data types, developing useful student survey 
questions, and understanding outlier and trend data. These impromptu mini-lessons are 
designed to deepen teacher expertise in data use. The school math coach has put together 
a set of mini-lessons that are ready as new data literacy topics emerge in teams. 

Element IV: Provide professional development on data-use knowledge and skills 

This high-achieving K-8 rural school does not have access to data coaches or external 
staff who can lead data practices. This school began intensive work on data analysis seven 
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years ago and focused on teacher teams using student data, including classroom (teach- 
er-developed) benchmark assessments and diagnostic and progress monitoring tools such 
as Dynamic Indicators of Basic Early Literacy Skills and Developmental Reading Assess- 
ment. Teacher teams are organized in multigrade groups (one team for grades K-2, one 
team for grades 3-5, and one team for grades 6-8). Teams meet weekly for 75 minutes to 
review current data and plan for next steps in instruction. Teams have another 45 minutes 
of scheduled time each week to talk about topics “other than data.” 

The principal articulates data use as a long-term process that requires developing skills and 
knowledge that build on what teachers know and can do. Each year, the faculty gives input 
to schoolwide goals that clarify expectations for school improvement in the area of data use 
and instruction. Schoolwide goals, over time, indicate deepening expectations for teachers 
to use data. In addition, teachers develop annual learning goals for their evaluation process 
that are based on their classroom data. The school leadership team, comprising one person 
from each data team, plus the school principal, finalizes schoolwide learning goals, and 
leads the professional development for all faculty. 

The multigrade data teams include subject-area teachers and special education teachers, 
working together to review the week’s data and plan classroom instruction and use of 
the daily “re-teach and enrich” time. Data analysis protocols are used to review student 
assessment data and help teachers document performance for each student against each 
standard. Teams also spend time on other tasks related to standards, instruction, and 
assessment, including clarifying learning goals, developing aligned assessment tasks, and 
aligning new content standards. 

While teachers in multigrade teams review classroom and benchmark data, the schoolwide 
leadership team attends to a close review of large-scale accountability data, identifying 
areas where the entire school can improve. Each year the school sets annual performance 
goals related to one or two high-leverage areas of focus. In past years this has included 
improving academic vocabulary, developing clear learning objectives, and providing useful 
feedback to students from each assessment. The principal, who heads the school-level 
data review, incorporates these goals into the data collection tool for the schoolwide walk- 
throughs, and teachers receive ongoing feedback on their progress towards these goals. 

To ensure ongoing professional learning for the whole school, the principal negotiates a 
change in district contract language each year whereby teachers agree to shift three days 
of professional development contracted for August to be distributed as monthly afterschool 
professional learning sessions. This change has been negotiated each year since 2006. 
Afterschool sessions, which take place following faculty meetings, allow for monthly 2.5- 
hour whole-school professional development sessions. 

Examples of schoolwide learning goals and related professional learning frameworks over 
several (nonsequential) years: 

2007/08 

Expectations for teacher learning 

• Data teams will learn and apply the use of three protocols for structuring the 
review of student classroom work. 
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• Data teams will be able to apply a process to review state assessment data in reading, 
writing, math, and science to establish curriculum focus areas for improvement. 

• Data teams will learn how to develop and apply SMART (specific, measurable, 
achievable, relevant, and time-hound) goals to address specific areas for improve- 
ment based on the state assessment data review. 

Whole-school professional development (approximately 30 hours, as determined by contract) 

• The school leadership team attends summer Critical Friends training to learn team 
dialogue protocols and bring them back to their school. In addition, the leadership 
team examines written resources to identify other protocols to meet goals for the 
coming year. 

• The school data team introduces and runs a protocol with teachers focused on 
analysis of student achievement results from state assessment data. The faculty 
develops data summary statements to highlight areas for further investigation. 
Data teams use these statements to focus areas for improvement in the coming 
year and to develop their SMART goals. 

• In monthly afterschool professional development meetings, teachers are intro- 
duced to a variety of data-use protocols from the Critical Friends training and from 
other sources that the leadership team has identified. Teachers apply these proto- 
cols with their own student work samples. After each session teams develop plans 
to apply specific protocols to address current team data-analysis needs. 

• Teachers participate in a voluntary full-year book study on Results: The Key to 
Continuous School Improvement (1999), by Mike Schmoker, at monthly breakfast 
meetings, with breakfast provided by the principal. 

• The School Leadership Team shares schoolwide state assessment data and cele- 
brates significant academic gains, particularly with Title I and special education 
students. Academic areas of focus for the next year are identified by teachers. 

• At the last meeting of the year, teachers share their SMART goals, describe the 
assessments used to monitor the goals, and outline how content and instructional 
practices were revised based on assessment evidence. Teachers work together to 
clarify what is different in practice and to provide input into the schoolwide learn- 
ing goals for the following year. 

2009/10 

Six faculty members attend a state summer conference on effective data use in schools. 
They use this information to scaffold professional development for the year. The leadership 
team focuses new learning on assessment literacy (how to develop effective assessments), 
data literacy (how to analyze multiple sources of data at once), and effective teaming prac- 
tices (how to ensure that time is spent addressing next steps in instruction for all students). 

Expectations for teacher learning 

• Teachers, supported by their data teams, will refine quarterly benchmark assess- 
ments that measure identified power standards, outlined in the district curriculum 
maps. 

• Teachers will use a range of evidence to determine next steps in instruction for all 
students. 

• Data teams will practice new data-analysis tools to analyze multiple sources of data 
and uncover underlying student-learning needs. 
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• Data teams will learn how to use various data-use protocols to ensure that, at each 
data meeting, they are better able to identify student-learning needs, and have 
time to identify key instructional actions for each of the “re-teach and enrich” 
levels for the following week. 

Professional development 

• The School Leadership Team runs two sessions on developing effective benchmark 
assessments. It covers topics such as aligning assessment questions with learning 
goals, developing and using distracter questions, and focusing on appropriate 
depth of knowledge. It provides a checklist for faculty to measure their fidelity to 
these topics; teams later use the checklist to review peers’ benchmark assessment 
development. 

• During afterschool professional development, teachers participate in “vertical 
alignment” work to clarify learning goals in key “power standards” by grade level. 
Teachers are given time to clarify changes to assessments based on their refined 
vertical alignments in each content area. 

• Summer conference attendees present to all faculty two new data-use protocols, 
the Data-Driven Dialogue protocol (from Love et al., The Data Coach’s Guide ) 
and the Grid protocol, which helps teams analyze proficiency data from multi- 
ple assessments. Teachers discuss the types of data analysis practices these tools 
support. 

• The faculty book study is Marzano (2007), The Art and Science of Teaching. Teach- 
ers spend 30 minutes at their monthly breakfast meeting reviewing high-lever- 
age instructional strategies, with a particular focus of how to apply instructional 
approaches in their weekly “re-teach and enrich” block. 

• The faculty invites a district office staff member to introduce the root cause analy- 
sis protocol, a process used to help uncover student learning issues. Two after- 
school sessions are spent learning and practicing the protocol. Teams discuss when 
they are most likely to use the protocol. 

• At the last meeting of the year, teachers share their SMART goals, describe the 
assessments used to monitor the goals, and outline how content and instructional 
practices were revised based on assessment evidence. Teachers work together to 
clarify what is different in practice, and provide input into the schoolwide learn- 
ing goals for the following year. 

2012/13 

At the close of the 2010/11 school year, the leadership team recognized that teachers were 
no longer applying the “re-teach and enrich” blocks. They realized that faculty members 
were using core instructional time to support individualizing instruction for all students. 
Rather than being “taught,” this process was a natural progression. As teachers became 
familiar with using data to inform instruction, they began to share that evidence with 
students and parents and began to develop instructional strategies to ensure students were 
fully aware of their next steps in learning. To support teachers in this transition, the lead- 
ership team looked closely at formative assessment literature, attended a summer confer- 
ence on formative assessment, and before the start of the 2012/13 school year, finalized its 
professional development and learning goals. 
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Expectations for teacher learning 

• Teachers will use evidence of student learning to give feedback to students about 
their progress on learning goals. 

• Teachers will learn strategies such as using portfolios, student self-assessment, and 
peer-assessment to help students better understand how to meet the learning goals. 

• Teachers, supported by their teams, will explore using model products to showcase 
excellent student work, and to clarify learning goals with students. 

• Teachers will continue to use differentiation and flexible grouping based on a 
range of data points. 

Professional development 

• As they begin the year, teachers indicate what has changed in their instruction- 
al practice over the past five years and where they see next steps for continuous 
improvement. Topics that come up align well with the leadership team planning. 
Teachers set and share new individual learning targets that align with schoolwide 
learning goals and outline expectations as to how they will measure their progress. 

• The principal leads the monthly faculty book study on Susan Brookhart s How to 
Give Effective Feedback to your Students (2008). Teachers agree to try each of the 
main strategies in the book over the course of the year and report their findings at 
book study meetings. 

• Two whole-school faculty sessions are devoted to using instructional rubrics with 
students. Teachers incorporate ideas raised in these sessions into their upcoming 
lesson plans, and the principal works with each teacher to clarify what he or she 
will see in daily walk-throughs as a result of this new work. 

• Teachers explore how to incorporate key features of formative assessment, includ- 
ing peer assessment and self-assessment strategies. Teachers model existing prac- 
tice and invite others to their classrooms to see strategies in play. All teachers 
agree to try a range of peer assessment strategies in the following month and then 
return with ideas on how to better manage the work. Teachers identify the kind of 
feedback they wish to receive on this work during upcoming peer walk-throughs. 

• Two faculty sessions are spent reviewing how to track formative assessment data 
and coming to schoolwide agreement about what data needs to be recorded and 
how. This continues to be an area of challenge for teachers, and many questions 
are raised about the expectations for schoolwide practice in this area. 

• At the last meeting of the year, teachers share their SMART goals, describe the 
assessments used to monitor the goals, and outline how they have revised assess- 
ment practices to include more classroom formative assessment practices. Teach- 
ers highlight how they revised instruction based on formative evidence. Teachers 
work together to clarify what is different in practice and to provide input into the 
schoolwide learning goals for the following year. 

Element V: Provide leadership to nurture a culture of data use 

This 200-student K— 8 school is in a historically rural area with recent development making 
it more suburban. The school principal was previously a literacy coach in the same school 
and has served as principal for approximately 15 years. The school has access to a district 
literacy coach and a district math coach, who each visit about once a week. A half-time 
assistant principal is also a half-time teacher at the school. 
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This school has always scored at or above average on state assessments until No Child 
Left Behind Act reporting began and results were disaggregated. Shortly after the act’s 
implementation, the district was identified as in need of improvement in math, mainly 
because of the scores of the special education and Title I populations. No K-8 schools were 
identified as in need of improvement due to small sample size. 

Following identification as a district in need of improvement, the district leadership team 
explored strategies for using data to guide instructional changes. Over the course of a few 
years (2005-08) the district implemented new interim assessments (such as the North- 
west Evaluation Association’s Measures of Academic Progress) to monitor student progress, 
introduced a new set of elementary diagnostic tools for math (First Steps in Math) with 
accompanying professional development for K-4 teachers, completed a K-12 curriculum- 
mapping project to ensure curricular alignment to math standards, and provided teachers 
with resources and time to develop common formative assessments in math. In addition, 
the district required schools to implement data teams by grade level, and lead teachers 
attended three days of professional development provided by the state on how to develop 
data teams. 

Between 2005 and 2008 district state assessment scores improved, but not remarkably. In 
2007 the principal at the aforementioned K-8 school decided to provide leadership in each 
building to support the numerous changes related to teachers’ expectations of data use. 
She did the following: 

• Implemented a schedule of quarterly data-review meetings with each teacher to: 
o Review individual classroom data. 

o Set academic goals for students. 

o Discuss the teachers’ use of instructional strategies to improve learning. 

• Established and led a school data leadership team to learn about key data-use prac- 
tices, including: 

o Assessment literacy (how to design common assessments, use distracters, and 
write assessment questions). 

o Data literacy (how to organize data for different purposes, how to prepare its 
own data for team meetings, how to look at multiple sources of data at once), 
o Collaborative teaming (how to select and use data analysis protocols, set team 
goals, facilitate team meetings, and use team norms). 

• Participated in all grade-level data team meetings. At each meeting, the princi- 
pal identified one area of progress to celebrate — often something that might have 
been “unsafe” to discuss (because it was often rooted in initially low performance) 
but that saw improved student performance through analysis and dialogue. 

• Re-structured professional development days to be informed by school perfor- 
mance data to include such topics as: 

o Strategies and protocols to analyze multiple data points to better understand 
student performance. 

o Curricular alignment based on recent performance data, 
o “Tuning” activities around assessment design and moderating the use of 
assessment rubrics. 

o Identifying and exploring ways to use exemplars of student work on key aca- 
demic standards. 

• Redesigned faculty meetings to focus on schoolwide evidence of improvement. 
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• Designed a schoolwide walk-through model for school leaders to “look for” evi- 
dence of data use in classrooms. Evidence collected from walk-throughs included: 
o Data clearly used for flexible groups, with group learning goals aligned to 

evidence. 

o Students able to articulate what the data showed about their learning and 
what they needed to do next. 

o Formative assessment strategies in place, including entrance and exit slips, 
observation checklists, and quick pre-assessments, 
o Students tracking their own evidence of learning through journals or 
portfolios. 

o Evidence of student learning displayed in classrooms. 

• Taught “intervention and extension” lessons in math. 

The principal s leadership was centered on two goals, both designed to ensure that teachers 
receive the support necessary to make significant and schoolwide improvements in learn- 
ing. First, the principal set clear expectations that teachers would be held accountable for 
using evidence of learning to guide instruction. Then, the principal provided a variety of 
supports to teachers to ensure that they would be able to apply and integrate new instruc- 
tional practices. Between 2008 and 2011, school state assessment scores in math increased 
more than 20 percent, perhaps in part because of the increased use of data strategies to 
inform daily instruction. 
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Handout 6.2: The Culture of Data Use Framework: Effective data-use practices 


Participate in the flow of information for evidence use 

Identify and invest in data management or warehouse 
systems. 

• Select appropriate data for classroom, school, and district 
use. 

• Limit the number of research-based programs schools can 
consider. 

• Provide data reports tailored to school specific requests. 

• Establish structures to encourage a strong bottom-up 
information flow from the school to the central office. 

Prepare data reports for school review. 

Identify staffing or personnel to support data functions (data 
entry, clerical support) that can be used or shared across 
schools. 


Provide resources and assistance to make meaning 
from data 

• Assist schools in finding ways to secure time for data analysis 
at the school level. 

Facilitate building-level discussions of progress overtime. 

• Support the idea that in most data-analysis practices, 
educators are juggling multiple forms of evidence at once. 

• Support the development of data teams, as they are shown to 
help make data accessible for teacher use. 

• Engage school staff to set performance goals and benchmarks 
that establish agreed upon learning outcomes and to create 
assessments aligned to those outcomes. 

• Organize and lead data retreats for school principals, both at 
their level and across levels. 

• Create consistent tools and processes (protocols) for data 
review. 


Communicate professional expectations for data use 

• Recognize teachers’ wealth of tacit knowledge as the starting 
point for data review. 

Model professional norms of behavior, including effective 
dialogue skills and use of the inquiry cycle. 

• Use evidence of improvement to provide feedback to system 
leaders. 

• Set the tone for accountability among staff members to ensure 
data review meetings are purposeful and lead to improved 
practice. 

• Write a clear data-use policy that outlines expectations for 
data use with students, teachers, parents, principals, and 
district office staff and that addresses challenges to effective 
data use. 

• Support consistent messaging to the community and parents 
about how the district engages with evidence of student 
learning at different levels of the system. 

Ensure safety and security of student information and 
evidence. 


Provide professional development on data-use 
knowledge and skills 

• Ensure that districts provide funding, expertise, and training 
for varied data-use practices across the district. 

• Ensure that districts provide professional learning for 
principals to use data at the building level and with teachers 
and teacher teams. 

• Encourage professional learning that focuses on building 
leadership and schoolwide expertise on using data to inform 
instruction. 

• Provide ongoing learning to a cadre of teacher leaders (often 
identified instructional specialists or coaches) who can lead 
data teams in grade-level and departmental configurations. 

• As teachers become proficient with analysis, professional 
learning shifts to supporting an expanded repertoire of 
instructional strategies to address data findings. 

• When training for new data systems, focus on the data; how 
it can be accessed, and what the system will do to support 
teachers and teams. 

• As much as possible, structure professional learning for data- 
use practices in small groups, using current district data. 


Provide leadership to nurture a culture of data use 

Ensure that leaders champion data analysis practices. 

Have districts address principal leadership in a structured way to build knowledge of data use and of how to lead data use in 
schools. 

• Provide opportunities for “tuning” practices to build consistency and coherence across districts and schools. 

Ensure that the central office requires principals to use data and holds them accountable for doing so over time. 

• Have the central office create and lead opportunities for principals to come together to share data-use practices at their building 
level and to analyze their data together. 

Have principals intentionally employ strategies to support data use to improve student learning in order to see teachers report 
better attitudes and more effective data use. 
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Handout 7.1: Changing expectations of data use: Teacher activities on the continuum 


Changing expectations of data use - 
teacher activities on the continuum 



STAGE 1 

Focus of work on 
school 

improvement 
planning using 
accountability data, 
and placement 
decisions 


*ne| Regional Educational 
Laboratory at EDC 
relnei.org 


STAGE 2 

Focus of work on 
curriculum 
development, 
student grouping, 
and attention by 
teachers on what 
to teach (ex. re- 
teach groups) 


EDC 


Learning 

transforms 

lives. 


STAGE 3 

Focus of work on 
comparative analysis 
of student growth, 
instructional changes 
based on how to 
teach - Entry into 
continuous 
improvement 


Major Partners: 

HAIR UJestEciT^ 


Material adapted from Means, 
Padilla and Gallagher, 2010 
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Handout 8.1: Culture of data use: District case study 


The following case study documents the development of a data-using culture in a suburban 
school district. The Keystone School District has seven schools (five elementary schools, 
one middle school, and one high school) and is located just outside a large metropolitan 
area. In 2012/13, K-12 enrollment was 5,600 students. 

Prior to being identified as a district in need of improvement in 2005, the Keystone School 
District primarily employed a site-based management model, where school leaders had 
significant autonomy in decisionmaking and school planning. Some school faculties were 
beginning to review and analyze student performance data. One school, for example, had 
adopted “critical friends groups,” where teacher teams analyzed and received feedback on 
student work samples and teachers discussed instructional changes to address student- 
learning needs. At another elementary school, faculty had been introduced to elementary 
literacy assessments through a Reading First grant and employed a new classroom literacy 
coach to support teachers’ analysis and application of student-learning data. Other district 
schools had used staff development days to review state assessment scores — days that were 
highly challenging because of the extreme limitations of the state’s technology dissemina- 
tion tools. Teachers left those sessions with evaluation comments such as “Just leave me in 
my classroom and let me be” and “What a total waste of time.” 

From 2000 to 2005 data use was primarily building-led: few clear messages came from 
higher levels about data-use practices, and teachers (with the exception of the Critical 
Friends Group site) largely had negative dispositions about data use. Meeting notes from 
that time show teachers, and to some extent leaders, discussing their frustration with 
state-level and large-scale data use. Notes include statements like “These meetings are all 
about accountability. . . . All this emphasis on data just takes away what makes me a good 
teacher — my creativity” and “This data is so old by the time we see it, it doesn’t tell me 
anything I don’t already know about my students.” 

The needs improvement status was based on low performance in English language arts by 
two subgroups: special education and English learner students. Many educators in the dis- 
trict were surprised because they had a sense that district students were high achieving. 
There were widely divergent responses across the district among educators. Some felt that 
the label was a good thing because it could spur the district to take action and support the 
poorest performing students. Others thought the designation did not offer anything of value 
and anticipated that it would cause more harm than good as teachers were asked to spin 
their wheels dealing with state mandates that were unlikely to impact instruction. These 
lines of demarcation hardened quickly. About a month after the designation, at a meeting 
facilitated by an outside consultant, the two camps were visible — to the point of sitting on 
different sides of the room. One group saw the data as a useful lever for change. The other 
saw the data as fundamentally flawed and wished to leave things the way they were. 

This polarization took place before the district superintendent could clarify a coherent 
message about the results or a clear plan of action. At the meeting, with the dividing lines 
clear, she conveyed a message that was repeated often over the next year: “We are not 
going to lose what is best in us, nor are we going to downplay these results. We are going to 
work together to make sure that our programs and instruction are supporting all students. 
It is no longer enough to say most of our kids are doing well. We will look closely at the 
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evidence we have, clarify where we can do better, and work together to figure out how to 
improve.” 

Over the next several years, the district office increased data-use practices at the school 
and classroom levels, established common practices for inquiry, and increased shared 
understandings for student-learning outcomes. Below are key highlights of district practic- 
es from 2005 to 2011. 

Group 1 

2005/06 

The district conducted a summer retreat for building leaders to establish common dis- 
trictwide expectations for data collection and use. Building teams developed data plans 
that were then calibrated by level (elementary or secondary). 

Another summer retreat was conducted for literacy teacher leaders to identify data expec- 
tations for literacy use in grades K-12 and streamline literacy data use across schools. This 
resulted in district plans to provide common training for teachers in literacy data-use 
practices. 

When 2006 state testing data were released, building leaders were taught an inquiry model 
of data use for large-scale data review. In a train-the-trainer model, building leaders were 
taught to implement this model with their faculty, and principals were tasked to complete 
this work with faculty at an “early release” professional development session. 

Principals came together after the common data-analysis protocol was used districtwide 
to share findings from teacher dialogue about state assessment evidence. The leadership 
team collected and shared the teachers’ data analysis findings with district content teams. 
In addition, the principals debriefed use of the common protocol and agreed on process 
revisions for the following year. 

After a district review of fall literacy data (which were triangulated from four sources), a 
literacy review team was established to study current districtwide literacy practices. The 
team’s recommendations were to align standards, instruction, and assessment and to estab- 
lish common K-8 literacy instructional practices districtwide. 

2006/07 

The district convened a team to begin clarifying how different types of assessments were 
used throughout the district. Using Rick Stiggins’ (2008) “Balanced Assessment Systems” 
as a model, it developed a data-using chart that was shared with all teachers. While teach- 
ers thought there was still a lot of data for them to review, they reported being pleased that 
some types of data analysis were “taken off their plate.” 

A teacher survey of data-use practice revealed three primary ideas that were discussed and 
adopted by the district leadership team. First, teachers asked for time to complete data 
review for their students. Second, teachers asked for more “useful” (that is, short-cycle and 
aligned) data. Third, teachers asked for opportunities to learn how to apply their data 
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results to classroom instruction. These three requests served as a guiding framework for 
building-level professional development and coaching. 

The district sent a team of 40 teacher leaders to a regional six-day professional develop- 
ment session focused on identifying essential standards and crafting “common formative” 
assessments aligned to those standards. Grade K-6 teachers focused on literacy, and grade 
7-12 teachers focused on their content teams. 

The assistant superintendent for curriculum and instruction convened all assistant prin- 
cipals for weekly meetings from January through May to redesign building-level structures 
to support data use and establish schedules to accommodate weekly collaborative meetings 
for every teacher. The assistant principals presented the new elementary, middle, and high 
school schedules to the superintendent and the district leadership team for review. All 
teachers have at least one hour weekly to discuss data in team meetings. 

Group 2 

2007/08 

During the summer of 2007, the district convened teacher leaders to develop an assessment 
calendar, by level, for the district. The team identified the range of data sources available 
for collaborative review in data teams, which current data were to be analyzed, and during 
which timeframe assessments would be administered and reviewed. The assessment calen- 
dar was used to guide the work of grade-level and departmental data teams. 

Also during the summer of 2007, approximately 200 teachers worked in small groups 
(assigned by level and content area) with content and assessment experts to learn common 
data-use protocols for use in data teams. The protocols and tools varied by type of assess- 
ment data, by level, and by content area. 

Following up on the six-day training the previous year, the district convened K-6 teacher 
leaders to align curriculum, instruction, and assessment in reading and writing. The team 
of 40 teachers was given monthly release time to clarify essential standards in literacy and 
to develop weekly “short-cycle assessments” that could be administered and reviewed in 
data teams. These assessments were developed by teachers and validated by an outside 
literacy expert. 

In a district leadership meeting in November, the principals noted that grade-level teams 
were “all over the map” in data use, with only minimal implementation of the protocols 
introduced in summer workshops. They decided to run monthly afterschool sessions to 
build consistency across the district in team data practices. The district offered a small 
stipend to teacher leaders who attended. During the sessions, teacher leaders asked that 
the principals not attend because they wanted to be free to talk about what is going wrong 
in their meetings. The principals chose to attend despite the request, with a message that 
they expect teachers to be in different places of adoption. These monthly afterschool ses- 
sions continued through the year. 
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The district convened a working group to explore the purchase of a new data management 
system. The group reviewed options and submitted a request to the superintendent. It was 
decided that that cost was too significant at the time. 

The superintendent convened a group of 22 teachers, leaders, and school board members 
to visit a district in another state that had had dramatic improvement after applying the 
same model of using short-cycle assessments aligned to essential standards. After a three- 
day site visit, the superintendent and school board convened a series of school and com- 
munity meetings at which teachers and principals discussed what they learned and shared 
an increasingly clear vision for the work. 

District leaders worked with various groups throughout the year to identify new assess- 
ments that could be used to more effectively inform students’ ongoing progress. New 
interim assessments were purchased for K— 8 literacy and math. 

2008/09 

Beginning in the summer of 2008, middle and high school content teams selected essential 
standards and crafted short-cycle assessments for each essential standard. Teacher teams 
met monthly throughout the year and included all content areas, including specialists. 
Assessments were designed to be used mid-unit and to support teacher dialogue about next 
steps in learning within a unit of study. 

In a district survey, the majority of teachers said they were spending, on average, an addi- 
tional one to two hours a week coordinating and preparing data. Focus groups were con- 
ducted to determine whether this activity was encroaching on instructional time. Teachers 
reported being frustrated because they were largely doing additional data work on their 
own time, and not all teachers were doing it. 

The district clarified a set of roles for data team members and developed job descriptions 
for each role. Teachers who took on the roles of “team facilitator” or “data expert” received 
extra professional development opportunities and a stipend for that work. The goal was to 
build capacity for individuals across the district to prepare data for team meetings and lead 
team protocols and dialogue, and to provide funding for those doing extra work. 

In November, the K-6 literacy team completed its work selecting essential standards 
and developing aligned common formative assessments. Teacher leaders (one for each 
grade-level team across the district) received training on effective implementation and use 
of these assessments. Teacher leaders brought these assessment tools and accompanying 
protocols to their teams. Teams were tasked with using the data from these common for- 
mative assessments to inform weekly instructional planning. Some grade-level teams begin 
to set up informal “re-teach and enrich” sessions as a way to re-group students for instruc- 
tion based on these assessment results. 

Newly purchased K-8 interim assessments in literacy and math were introduced with 
much difficulty. Teachers reported they were not clear about how to use the assessments 
and were confused by different messages from academic coaches and principals. 
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Literacy coaches worked with an external consultant to explore ways to help teachers “tri- 
angulate” data during team meetings so they could integrate the new assessment data. 
New expectations were placed on teams to review multiple data sources, and the coaches 
identified this as an area for skill development. The coaches developed a fairly complex 
template that aligned data by standard and student, but teachers dismissed it as too time 
consuming to use in a data meeting. 

The district’s state assessment scores improved significantly over two years, and the district 
was removed from “needs improvement” status. The district leadership team congratulated 
teachers for leading and guiding student improvement through evidence use. 

Group 3 

2009/10 

The District Leadership Team met over the summer to outline next steps in data-use pro- 
fessional learning for their teachers. It reported overall growth in teams using data effec- 
tively, with pockets of excellence and also some examples of limited practice. Two new 
approaches for professional learning for data use were instituted this school year. The first 
was to ensure every teacher has a professional growth goal related to the building’s data 
goals. The second was for principals to visit each team monthly to support the teams’ 
taking next steps in their learning. 

Official “re-teach and enrich” groups were established in all five elementary schools. Sched- 
ules were revised to ensure a 120-minute literacy block, 30 minutes of which was used for 
re-grouping across a grade level to focus on targeted student needs. K— 6 data teams focused 
almost exclusively on preparing and analyzing data from weekly, short-cycle assessments 
and on planning for the re-teach and enrich blocks. 

Additional time was devoted to teacher leaders at the middle and high schools to com- 
plete content-area short-cycle assessments that could be used mid-unit to guide instruction. 
The process of sharing these assessments and implementing them with teacher teams was 
largely hit or miss, with some positive adoption and some negative adoption — where the 
assessments appeared to be used to punish students rather than to guide next steps in 
instruction. Teacher leaders reported being frustrated. 

After successfully offering stipends and supports for teachers to take on data team roles the 
previous year, the district offered the same support — but this time required that teachers 
take turns leading different aspects of the data team work. There was significant backlash, 
with teams reporting that they had learned various tasks and now the district was making 
them take on new roles “for no good reason.” The district message — that a goal of this 
support is to build capacity widely — seemed not to have been understood this time around. 

Parents were complaining on Facebook that teachers were “never there” and “always at 
training.” The superintendent and school board crafted a shared response. At a public 
meeting, the school board chair documented improvements in student learning over time 
and talked about the impact of new instructional techniques on his daughter’s learning 
and about the power of this work to improve learning outcomes for all children. The board 
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asked parents to commit to another year of “intensive time” for teachers to develop effec- 
tive assessment and data practices. 

Annual state assessment data continued to improve, with one elementary school showing 
significant gains in English language arts for lower achieving students. Credit was given 
largely to a site-based response to intervention model that had been introduced at that 
site about 18 months earlier. Other principals were encouraged to learn more about this 
research-based model and consider implementation. 

2010/11 

Summer work focused on teachers developing strategies to “extend” learning for students 
doing well on short-cycle assessments. Teachers developed learning progressions for literacy 
standards (in grades K-6) and unit standards (grades 7-12). Teachers worked with consul- 
tants and internal teacher leaders to develop rigorous instructional strategies to address 
students scoring well on specific standards. Sample lesson plans and instructional guidance 
were developed to lead more effective “enrich” lessons. 

The state sponsored a summer workshop on implementing professional learning commu- 
nities, and a large group of K-12 teachers attended. They returned to the district saying, 
“We’re already doing this,” but they added a set of protocols to their online data toolkit 
that were introduced to teams the following year. 

At the beginning of the school year, the District Leadership Team stated there would be 
no more new data expectations introduced during the academic year. On the first day 
of school, the superintendent stated this in the teacher welcome meetings. The superin- 
tendent cataloged the current data-use expectations at the individual, team, school, and 
district levels and asked teachers to “keep getting better” over the course of the year. Wide- 
spread delight greeted this announcement, especially the news that several data-analysis 
expectations had been shifted from teacher teams to building-level analysis. 

Principals continued to work with data teams monthly through the year to clarify out- 
comes and support team learning. 

Principals continued to work with each teacher to set an annual learning goal focused on 
data use. Principals reported that many teacher goals had moved from a focus on team 
analysis of data to a focus on analysis of their own classroom data. 

After intensive work supporting data teams, state assessment results showed dramatic gains 
in high school (for the first time) in English language arts and math scores. All other levels 
showed some improvement, particularly among lower achieving students. However, the 
percentage of students at the highest proficiency level decreased in all K-8 schools. 

In spring 2011 principals met to explore ways to share excellence across schools and grade 
levels and to enhance widespread capacity to use evidence. They decided to celebrate their 
best practices. To do this, they planned a K-6 “data-driven instruction fair” to take place 
during the first two days of a back-to-school workshop in August. Sixty teacher leaders 
were identified by their principals and coaches as having outstanding examples of best 
practice in 15 areas. These teachers were convened in May 2011 and worked in teams of 
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three to four, by area of excellence, to develop a 50-minute presentation to their peers. 
Examples of topics included “Using evidence of learning during peer assessment” and 
“Incorporating individual student evidence during conferring” The presenting teachers 
were given stipends to work during the summer to refine their presentations. The work- 
shop days were rated as the most successful professional learning ever to take place in the 
district, and since each teacher only attended four of the sessions, the work continued the 
following year in various ways. 

The Keystone District knows this work is ongoing and that it has more work to do. Over 
multiple years it has accomplished building an emerging culture of data use, in which, even 
when no one is looking, teachers and leaders know which data to use, know when to use 
data, and know how to use data to make decisions about teaching and learning. 
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Handout 9.1: Discussion protocol: Identifying 
implementation strategies for a culture of data use 


1. Working with colleagues, identify one of the five framework elements to work with, in 
order to develop some initial ideas for implementation strategies- (5-10 minutes) 

Framework element/focus area: 


2. Brainstorm and list three or four barriers that impact implementation in the frame- 
work element that you selected in item L (5 minutes) 

3- In handout 9-2 review only the framework element that you identified in item 1. 
Discuss and then list one or two more barriers from the handout that apply in your 
setting. (5-10 minutes) 


4- In handout 9-3 review and discuss the strategies designed to support consistent imple- 
mentation for the framework element that you identified in item 1- Then address the 
following questions- (10-15 minutes) 

a. How do these examples build consistent data-use practices? 

b. Would any of these example practices help address the barriers you listed above? 
How? 

c- How are these examples similar to, or different from, strategies you’ve tried in your 
setting? 

Note key points from your discussion: 


5. In handout 9-4 review and discuss the guidance for the framework element that you 
identified in item 1. Then address the following questions- (5 minutes) 

a- What additionally do these examples add to your discussion of this framework 
element? 

b. In what ways do these examples provide further ideas around developing consis- 
tent practices that further data use work in this framework element? 

Note key points from your discussion: 


6- What are some thoughts related to next steps that can support deepening a culture of 
data use in your setting? What are one or two high-priority ideas to move forward from 
this discussion? 


H-46 


Handout 9.2: Barriers to a culture of data use, by framework element 


Participate in the flow of information for evidence use 

• Data management systems are cumbersome, inefficient, and 
frustrating for teachers to use. 

• Technical limitations of data systems suppress data use. 
Educators go to great lengths to compensate for a lack of 
integration with data systems and might even have built their 
own systems to address the lack of a district system. 

• Teacher leaders take on the task of coordinating and preparing 
data for teacher team meetings, usually during their own time. 
Educators can become frustrated with the amount of time it 
takes to access and analyze data, most frequently because of 
a lack of system integration. 


Provide resources and assistance to make meaning 
from data 

Data are used, but there is little evidence of collegiality. 

• Protocols for data use are followed but only at the most 
procedural and basic level. Team dialogue does not focus on 
dialogue about instructional change. 

• Structures and protocols for collaboration are not used or 
understood. 

Teachers who believe in data-use practices do it on their own 
time — before school, after school, or at lunch. 

• Ineffective access to data prevents collaboration and dialogue 
about instructional practices. 


Communicate professional expectations for data use 

Data are mistrusted and are seen as a compliance tool rather 
than an instructional support. 

* Teachers have competing time demands for data use and 
analysis. 

* Formal district policies around data use do not exist. 

Educators appear to be negative about data use, but deeper 
questioning reveals that it is not the idea of data use in itself 
that concerns educators. Rather, it is the perceived difficulties 
that arise with data use — the amount of time required, the 
lack of access to data, and so on — that prompt the negative 
responses. 

Different data-use expectations are in play across district and 
schools. 


Provide professional development on data-use 
knowledge and skills 

• Professional development occurs in large group settings and 
does not address skill development using teachers’ own data. 

• Professional development takes place in the early stages of 
data use, but as teachers shift to more challenging practices 
(for example, interpreting evidence to use during instruction), 
professional learning is no longer focused on data use. 
Evidence about teachers’ current practices and learning needs 
is not collected, and professional learning is not aligned to 
teacher learning needs. 

Different messages from competing professional development 
providers hinder coherence and application of new practices. 


Provide leadership to nurture a culture of data use 

Time is not provided for collaborative review of data, or the time that is available is barely monitored for effective practices. 

• The culture is one in which teachers do not feel safe revealing where they need to improve practice, and leaders (inadvertently or 
not) punish teachers for sharing areas of weakness or concern. 

* Decisions at the district or school level are made “from the gut” and do not model effective data-use practices. 
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Handout 9.3: Examples of policy and guidance to 
support a culture of data use, by framework element 


Participate in the flow of information for evidence use 

* Written expectations for data use that show which practices 
align with improved practice and document the shift away from 
an accountability-based data-use approach. 

• Written clarification identifying which users of data are meant 
to answer which questions. 

• Calendars and timelines of district data-use expectations by 
grade. 

Description of district data systems to clarify functions and 
uses. 

* Description of district data systems that describe how they 
support and align with the everyday work of educators. 

Provide resources and assistance to make meaning 
from data 

• A school calendar documenting scheduled time to analyze 
evidence. 

A common location (computer drive) with written protocols for 
using evidence during team meetings. 

• Written role expectations for data teams with specific team- 
member functions outlined. 

• A schedule with opportunities for calibration. 

Agreed upon norms for data use posted and reviewed at all 
meetings. 

• Written documentation of how coaches or instructional 
specialists will support collaborative data-use practices. 

Communicate professional expectations for data use 

• Written guidance about the focus of data use as designed to 
support all students. 

• Written communication highlighting how the focus on all 
students will raise the achievement of struggling learners 
through increased differentiation and personalized student 
supports. 

• Written expectations about annual “products” that include 
student evidence, including written guidance for use of 
evidence at parent-teacher meetings, portfolios, and 
information that is documented across years. 

Provide professional development on data-use 
knowledge and skills 

• An annual schedule of professional learning for data use, 
including formal, informal, large-scale, team-based, and daily 
learning, focused on common learning goals schoolwide, 
and including individual (or team) areas of focus for teacher 
learning. 

• Written structure to document teacher learning goals regarding 
using data. 

• Protocol for principals to review teachers’ practices with 
various types of data use (including both schoolwide and 
classroom uses) and to outline next steps in their individual 
learning. 


Provide leadership to nurture a culture of data use 

Job descriptions that capture the role of data-use leaders throughout the district: principals, teacher leaders, data team leaders, 
district leaders. 

• A hiring protocol outlining expectations or activities that show facility with data use. 

Documentation related to how leaders learn data use overtime, with increased expectations outlined overtime. 
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Handout 9.4: Examples of district and school guidance 
to support a culture of data use, by framework element 


Element I: Examples of participating in the flow of information for data-use policy and guidance 
District implementation of student data systems 

Excerpted from Means, B ., Padilla, C., & Gallagher, L. (2010). Use of education data at 
the local level: From accountability to instructional improvement. Washington, D.C.: U.S. 
Department of Education, Office of Planning, Evaluation and Policy Development, httpfilwww. 
ed.gov/about/officesllist/opepd/ppsslreports.html#edtech. 

One large, suburban district has spent over a decade trying to make data available to its 
schools. The district’s development of a data system to support these efforts has been an 
iterative process that has included ongoing development of new data systems to replace 
outdated ones and the incorporation of new technology as it is developed. Their current 
student information system (SIS) was developed locally over seven years ago. When the 
system was first implemented, information from the data warehouse was exported into the 
SIS to make it more accessible to teachers. Unfortunately, teachers did not use the system 
because it did not meet their needs and was not user friendly. The same result occurred 
with a commercial assessment system. According to the executive director of information 
systems and support: “Technically you can make most anything work. [We had a] good 
vision of where we wanted to go, but when it came down to sitting down with the individ- 
uals [vendors], it was difficult to get an end product. ... It took from 1999-2000 to 2008 to 
get a tool that was teacher-friendly. That was our objective up front, but now it has finally 
evolved into a true teacher tool. The district has always had lots of data, but there was 
a challenge turning data into information.” From these experiences district staff learned 
that they needed to involve subject-matter experts to develop data system solutions. 

Through an ongoing process of soliciting feedback from users, this first district has begun 
to replace the current SIS with a new system that utilizes commercial software to make 
data more intuitive and teacher-friendly (teachers have drill down capabilities for their own 
classes and individual students). The district maintains two commercial data warehouse 
systems — one is a legacy system used for state reporting and keeping records of students with 
special needs. Within the last two years, the district has acquired an assessment system that 
was implemented districtwide during the 2007-08 school year. This system contains district 
benchmark data that is made available to teachers five times a year within 24 hours after 
test administration. The assessment system also contains links to district standards, pacing 
guides, and an item bank aligned with state tests. In 2006-07 the district implemented a 
locally developed Web-based portal so that teachers can access data anywhere any time; the 
portal also helps to support the interoperability of the districts’ various data systems. Over 
the next few years, the district will continue to manage the transfer of student data to its new 
SIS, try to improve the linkages between the assessment system and the curriculum manage- 
ment system, and link electronic teacher gradebooks with the SIS. The district will also try 
to find a way of storing portfolio-based information that current data systems do not support. 

One of the strengths of the second large district has been its capacity to build its own data 
systems customized to the needs of district and school staff. The district’s data warehouse 
was locally developed. Using this system, district and school staff can generate standard 
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accountability reports or district and school report cards, and record daily class attendance 
and disciplinary actions. The data system is the primary tool used by schools for drafting 
and revising school improvement plans. Their data warehouse has the capability to link 
student performance data to student subgroups so that school and district staff are able to 
disaggregate data to perform different data queries, and student performance data can be 
linked to specific teachers, teacher characteristics, and specific instructional programs so 
that staff can examine student performance in different classrooms or programs. When 
the district found that their off-the-shelf assessment system did not fully meet their needs, 
they decided to develop their own. District staff are working in conjunction with a com- 
mercial firm to design an assessment system that includes an item bank (items that are 
aligned with state standards and the district pacing guides) and provides links to instruc- 
tional resources. The assessment system will be linked to the data warehouse to support 
aggregating data and longitudinal analysis, and a new interface will highlight for teachers 
and principals where potential problem areas might be and let them drill down to access 
the relevant information. This district is also in search of a data system that is capable of 
storing student portfolios and other nontraditional forms of data. 

Element II: Examples of communication policy and guidance 

District guidance: Organizing scheduled data reviews 


This guidance brief supports school staff to schedule time for frequent data meetings to ensure 
that teachers have structured time during the school day to review student performance data. 


Data review 
task 

Data for analysis 

Key players 

Focus for analysis 

Schedule } 

Overall 
achievement 
and trends 

• Graduation rates 
Attendance 

• State assessment 

Entire staff, 
school data 
team 

Program review 

• Curricular review 

• Patterns and trends of overall student learning 

• Areas of focus for professional development 

Beginning and 
end of year 

Student 

growth 

• District benchmarks 

• End of course assessments 

• Common assessments 

• Student work 

Teacher and 
administrator 

• Patterns of student performance over multiple 
years 

• Instructional strategies to address specific 
learning needs over time 

Professional growth for teachers based on 
student learning 

Three times 
per year 

Whole 
class and 
individual 
performance 

• Common assessments of 
student work 

• District benchmarks 

• Behavior data 

9 Competency assessments 
(high school) 

Grade-level 
(elementary) 
or department 
(middle or high 
school) teams 

• Clarify next steps in learning for groups or 
individual students 

• Share instructional practices 

* Group students for intervention 

* Response to Intervention 

Every two 
weeks (team 
time) 

Whole 
class and 
individual 
performance 

• Student work samples 

• Common assessments 
- Unit assessments 

• Competency assessments 
(high school) 

Elementary 
learning teams, 
or secondary 
department 
meetings 

• Patterns and trends in performance 
Instructional strategies to address needs 

• Explore strategies for students to set learning 
goals based on data 

• Develop classroom and formative assessment 
tasks 

Elementary: 
two weeks 

High school: 
four weeks 

Student goal 
setting 

• Student portfolios 

• Student work samples 

• Common assessments 

• District benchmark 
assessments 

Teachers and 
students 

• Goal setting 

• Portfolio review 

• Strategies for success 

Elementary: 
two weeks 

High school: 
four weeks 


Source: Adapted from National Staff Development Council (2009). 
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Planning for a coherent assessment system 


Assessment 

type(s) 

Assessment tools 

Expectations for users of data 

Teachers Students School/district 

Focus for 

professional learning 

Daily: 

Classroom 

formative 

Questioning 
White boards 
Exit tickets 

Observational records 

Monitor student 
understanding of 
learning goals and revise 
instruction based on data 

Self-monitor 
progress 
toward learning 
goals and make 
adjustments as 
needed 


Use data to set 
learning goals with 
students 

Use data “on the fly” 
to change instruction 

Develop flexible 
groups 

Track formative data 

Periodic: 

Diagnostic 

assessments 

and progress 

monitoring 

tools 

Dynamic Indicators of 
Basic Early Literacy 
Skills 

Developmental Reading 
Assessment 

Aimsweb (math) 

Place students into 
intervention groups, using 
the district response to 
intervention mode 

Increase understanding of 
student-learning needs 

Self-monitor 
progress 
toward learning 
goals 

Understand 

placement 

decisions 

Provide early 
identification of 
students who 
require targeted 
interventions 

Diagnose individual 
knowledge and skills 

Group students for 
instruction 

Establish 

intervention groups 

Identify next steps 
for learning 

Weekly or 
biweekly: 
"Common” 
assessments 
designed by 
grade level 
and aligned to 
standards and 
calendar 

Weekly assessments 
developed by teacher 
teams 

Evidence reviewed in 
grade-level data teams 
to guide instruction 

Check individual and class 
progress against learning 
goals 

Guide dialogue about next 
steps in instruction 

Check progress 
toward learning 
goals 

Clarify what 
has been 
learned and 
what comes 
next 

Identify students who 
require additional 
supports and 
students who are 
ready to move to 
higher level work 

Identify patterns in 
proficiency 

Placement 

considerations 

Guide re-teach and 
enrich groups 

Periodic: Unit 
testing 

High school 
competency 
assessments 

End of unit tests 

High school competency 
assessments 

Determine grades and 
promotion 

Gauge progress 
toward meeting 
standards and 
share with 
parents at 
conferences 

Conduct department- 
and grade-level team 
analysis of mastery 

Clarifying grading 
and reporting 
for formative 
versus summative 
assessments 

Communicating with 
parents 

Quarterly: 

Benchmark 

assessments 

Fountas and Pinnel 
Benchmark Assessment 

District benchmark 
assessments in math, 
science, and social 
studies. 

Northwest Evaluation 
Association assessments 
(grades 6-12) 

Analyze which students 
need additional 
interventions or targeted 
services and which can 
benefit from advanced work 

Analyze learning trends 
for individual and groups 
of students compared to 
national norms 

Gauge progress 
toward meeting 
standards and 
share with 
parents at 
conferences 

Analyze trends in 
student performance 
overall and across 
grades and buildings 

Identify which 
students are meeting 
the standards and 
which are not 

Identify area of 
student growth 

Use effective 
instructional 
strategies 

Identify patterns in 
proficiency 

Placement 

considerations 

Annual: State 
assessment 

New England Common 
Assessment Program 

Analyze curricular focus 
areas for improvement 


Identify broad 
areas of strengths 

Train leadership 
to use common 


Align curriculum to state 
standards 


and weakness in 
the curriculum, 
determine degree 
of progress, and in 
which areas students 
are not meeting 
standards. 


protocols at the 
school to support 
effective use of 
state assessment 
evidence, including 
patterns of student 
achievement, 
instructional 
programs, additional 
knowledge, skills, or 
resources for staff 
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Element III: Examples of making meaning from data policy and guidance 


Common formative assessment tuning protocol 
How to use this tool: 

• Select a recently implemented common formative assessment task. 

• Complete this reflection tool individually. 

• Bring your completed reflection tool, the assessment used, and your planning doc- 
uments to your team meeting. Discuss your reflections with colleagues. 

• Clarify what worked well with this common formative assessment and what you 
might like to change. 

Upon reflection, to what extent did this common formative assessment: 


Reflections prior to group review 


Very well 


Not at all Group discussion notes 


1. Clearly align with the learning goal? 


4 

3 

2 

1 

0 


Comments: 


2. Address the knowledge and skills 
that are documented in the learning 
progression? 

3. Identify and elicit the content or skills 
where students have misconceptions? 


4 

3 

2 

1 

0 


Comments: 


4 

3 

2 

1 

0 


Comments: 


5. Provide the teacher with useful (that 
is, new) information about each student’s 
level of understanding? 

6. Provide the teacher with useful 
information for adjusting instruction? 


4 

3 

2 

1 

0 


Comments: 


4 

3 

2 

1 

0 


Comments: 


7. Lead to a change in instruction? If so, 
what changed? 


8. Lead to an exchange of feedback 
between student and teacher? 


4 

3 

2 

1 

0 

Comments: 

4 

3 

2 

1 

0 


Comments: 


After shared review of responses, discuss the overall trends. 

• If your common formative assessment did provide useful guidance to the teacher, 
what might you take away regarding future strategies for developing or implement- 
ing common formative assessment tasks? 

• If your common formative assessment did not provide useful guidance to the 
teacher, what might you change in this assessment? What might you want to do 
differently as you develop or implement your next common formative assessment? 

Document your feedback, and share key findings from your discussion with your team and, 
through team notes, with your administrators. 


H-52 




School district collaborative assessment project 


Professional learning team member roles 

• Come prepared to share data and student work samples. 

• Discuss and analyze data as they relate to the standard. 

• Group students for interventions and extensions. 

• Develop lessons for interventions and extensions. 

• Adhere to established data meeting protocols. 

• Learn to use multiple forms of data, including common formative, benchmark, 
and diagnostic tools. 

• Focus on learning goals. 

• Know when to ask for support. 

• Focus on continuous improvement to better support student learning. 

• Co-develop lesson plans to include new instructional techniques. 

• Provide feedback on common assessments for annual revisions. 

Professional learning team roles eligible for “pay for performance ” 

Facilitator. A team facilitator is a “neutral” person who supports the team in using and 
developing process skills for more effective and efficient use of team time. Individuals in 
this role provide advance preparation for meetings, learn and practice meeting facilitation 
skills, and work to keep the team focused and on task throughout the meeting. Four days 
of afterschool and summer training will be provided to all team facilitators, with reim- 
bursement at the district stipend rate. Primary facilitation tasks are: 

• Ensure that materials are organized and prepared before the meeting. 

• Create a risk-free environment that encourages collaboration and makes room for 
mistakes. 

• Keep discussions focused on standards, assessment, and improving student 
achievement. 

• Encourage curriculum and assessment to be aligned and consistently implemented. 

• Pose difficult questions and offer opportunities for courageous conversations. 

• Facilitate the team’s identification of teaching strategies that will be used during 
classroom instruction. 

• Facilitate the team’s identification of teaching strategies that will be used during 
interventions and extensions time. 

• Keep the moral imperative and greater purpose in the forefront of all work. 

• Allocate equitable and appropriate time, training and resources to support profes- 
sional learning team work. 

Data analysis leader. The data analysis leader demonstrates the ability to lead teams in 
analyzing multiple sources of data to identify improvement needs, symptoms, and instruc- 
tional issues and in working with teachers and school leaders to determine improvement 
goals based on that data. Four days of afterschool and summer training will be provided 
for all data leaders, with support in using the district information management system, 
using excel spreadsheets, and preparing data displays for team analysis. It is hoped that in 
future years training can be continued on data literacy for leaders in this role. Primary data 
analysis leader tasks are: 

• Collect and record assessment data prior to professional learning team meetings. 

• Provide appropriate data in a user-friendly format and in a timely fashion. 
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• Share whole group, subgroup, and item analysis with professional learning team 
members, 

• Lead professional learning team in analyzing classroom- and grade-level results to 
guide instructional decisions. 

• Share data with school leadership or other teams for further analysis schoolwide. 

• Assist team in generating revisions in classroom instruction based on analyzed 
data. 

• Assist team in monitoring goal progression throughout the school year. 

• Display data in a visible way so that data are transparent and understandable. 

• Support the team in making data queries where needed. 

Note taker . The note taker maintains team notes, records data findings/data statements, 
and keeps an accurate record of instructional decisions the team has discussed to address 
findings from the data. The note taker distributes notes to the team and maintains a team 
electronic notes folder that is easily accessible by any team member. Primary note taker 
tasks are: 

• Take notes during professional learning team meetings. 

• Distribute notes to team members regularly. 

• Maintain accurate records of data analysis reports and protocols. 

• Maintain accurate records of instructional practices that are chosen based on data. 

• Ensure records are kept in a place where other teachers are able to readily access 
for review. 

• Provide regular communication with building leaders about team progress. 

• Work with team to maintain an instructional binder for each essential standard in 
literacy and math. 
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Professional learning team common formative assessment review 


Team data analysis protocol: Five-phase data review 


Stage 4: Infer, question, 

Stage 1: Predict Stage 2: Go visual Stage 3: Observe and analyze and clarify 

(5 8 minutes) (5 8 minutes) (10 15 minutes) (15 minutes) 


The goal during this time is 
to think about the team’s 
collective knowledge and 
experience of this standard 
prior to collective data review. 

Key questions: 

What are we thinking about this 
standard? Are there noticeable 
misconceptions or challenges 
students had learning this 
specific standard? 

Conversation starters include: 

• I predict... 

I assume... 

• When I taught this standard I 
was wondering... 


Each team member reviews the 
data quietly for a few minutes 
and then shares key facts from 
the data. 

Key questions: 

What do you notice about these 
data? What “data statements” 
can be made in reference to 
these data? What important 
points seem to “pop out”? 

Conversation hints: 

• Share only factual 
statements 

* Resist saying “because” 


During this collective data 
analysis, team members look 
for patterns and trends in the 
data. 

Key questions: 

What patterns and themes are 
emerging from the data? How 
does this information compare 
to the predictions the team 
made earlier? 

What do you notice about 
trends or outlier information 
from this data set? 

Conversation starters include: 

I notice that... 

• I see that... 


Team members move 
from looking at causes to 
determining instructional 
actions. 

Key questions: 

In what ways do these data 
offer suggestions for next 
steps to extend student 
learning? In what ways do 
these data offer suggestions 
for next steps to provide 
interventions for students who 
require it? Which instructional 
strategies might we apply? 

Conversation starters include: 

• A clear student-learning need 

is... 

The data suggest that... is a 
specific area of weakness... 


Notes: Notes: Notes: Notes: 

Areas of learning to revisit: 
Areas of learning needs to 
extend: 


Stage 5: Action planning 


Instructional actions for specific identified needs for “revisiting” key standards: 


Instructional actions related to next steps in learning for students who are meeting key standards: 
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Qrade 1 professional learning team meeting notes template 


Date: 

Facilitator: 
Note taker: 
Timekeeper: 


1 Attendee 

Please initial 

Attendee 

Please initial j 

Teacher name 

Specialist name 

Teacher name 

Literacy coach name 

Teacher name 

Administrative liaison name 

Teacher name 

Special education teacher name 


Standard: 


Stage 5: Agreements and recommendations for intervention and extension groups 


Action plan: Teacher assignments and next steps for re teach. Recommendations to response to intervention team. 


Copy of agreements and recommendations sent to: 

Team administrative 
liaison 


Shared Literacy 

drive coach 
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Element IV: Examples of professional development policy and guidance 


Rural middle school data team : Embedded professional learning 

This plan represents a first-year of implementation of a rural middle school data team. This 
team was supported by an outside consultant, but all embedded professional learning and 
faculty presentations were led by the data team members. 

Schedule of data analysis activities and teacher and team support 

Team goal for the academic year: Provide “embedded” professional learning to all teachers 
by leading data analysis sessions, introducing protocols for data review, and providing strat- 
egies to use evidence of learning to guide student academic goal-setting. 

To meet this goal the data team will: 

1. Learn and apply protocols and tools for looking at evidence of student learning — par- 
ticularly in helping teachers look at evidence of learning during common planning 
time meetings. 

2. Be out ahead around data analysis and interpretation — including knowing how to 
use existing data sources (Northwest Evaluation Association, Performance Pathways, 
Aimsweb, state assessment) and how to support teachers during common planning time. 

3. Define and help frame consistent schoolwide data analysis practices, including under- 
standing a range of analysis protocols to employ with different types of data (student 
work, unit assessments, interim assessments, diagnostic assessments, and large-scale 
assessments). 

4- Make sure common planning teams have the information they need to do data 
analysis, to establish student-learning goals based on data, and to begin to use data to 
inform daily instruction. 

Data team processes: Supporting data use in common planning time 

The data team agreed that there would be four data-focused engagements with common plan- 
ning teams through the year, in which common planning teams would review data using a 
common protocol. This work will be led by data team members. Each of these data-use activities 
will help teachers learn how to analyze and use different types of evidence of student learning. 

The four data review processes are: 

1. Diagnostic assessment data review (October). 

2. Interim assessment data review (November and April). 

3. Analysis of student work on research writing rubric (January). 

4. State assessment data review (March). 
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Annual calendar of data team activities 

(Meetings are scheduled every two weeks; team members are paid a stipend.) 

September 

• Plan for diagnostic assessment review with common planning teams. 

o Complete advance review of classroom data sets for new math diagnostic data 
tool. 

o Identify content strands on which to focus gradedevel review: number sense, 
measurement, operations, patterns and relationships, data (streamline), 
o Agree on protocol for data review and analysis (select from common planning 
team materials) and clarify process for moving from analysis to instructional 
actions. 

o Prepare class datasets for review in common planning team. 

October 

• Debrief meetings with common planning teams related to review and analysis of 
diagnostic assessments. 

o Identify and record what worked well and where to improve in “train the 
trainer” model. 

• Plan for interim assessment review with common planning teams, 
o Clarify which interim assessment reports to review with teams. 

o Prepare goal sheets for individual students to set learning goals based on 
interim data. 

o Practice using interim online materials and resources (with tech coordinator), 
o Identify goals for common planning teams and clarify protocol and process to 
complete interim assessment review that leads to students using interim data 
to set learning goals. Share with common planning teams in advance. 

November 

• Finalize materials for review of interim assessment data with common planning 
teams. 

• Work with common planning teams to complete review of interim assessment 
data process. 

• Develop a strategy to collect examples of research and reports from across grade 
levels for review at common planning team meetings in January. 

• Identify one of the Looking at Student Work protocols from the common plan- 
ning team binders to implement in January common planning meetings. Practice 
completing the protocol and debrief to deepen leadership and training approach. 

December 

• Finalize materials for review of student work and writing rubric with common 
planning teams. Clarify approach to capture schoolwide findings across teams to 
guide next steps in cross-curricular writing instruction. 

• Debrief work with common planning teams related to review of interim assess- 
ment data. In particular, clarify how common planning teams are using interim 
data to establish student-learning goals. 

• Continue to develop data team skills around assessment design and use. Invite dis- 
trict math consultant to share approach to developing common math assessments 


H-58 


for grades 1-5- Discuss data analysis protocols that have been introduced with ele- 
mentary grade-level teams to review common assessments in math. 

• Explore how colleagues at other schools are developing and using common assess- 
ments and provide some recommendations to transformation team about how to 
develop and use common assessments at the middle school 

January 

• Conduct analysis of student work and writing rubric with common planning 
teams. 

• Complete analysis of findings across common planning teams to begin to identify 
schoolwide focus for next steps in writing instruction. Discuss with principal and 
prepare materials to share at faculty meeting for review. 

• Debrief analysis of student work and writing rubric, noting process strengths and 
areas where faculty needed additional support. 

February 

• Prepare and bring information to leadership team regarding what the data team 
has learned about schoolwide data review practices through common planning 
teams. 

• Review annual state assessment data for identification of schoolwide trends, sub- 
group performance, and areas of curricular focus. 

• Prepare state assessment data into charts, graphs, and tables that can be used in 
faculty data review. 

• Identify a data-analysis protocol to use at a whole-school meeting to have faculty 
engage with state assessment data and determine key data findings. 

March 

• Report to school leadership team about key state assessment findings and initial 
review of next steps based on data. 

• Work with faculty to review state assessment data and set schoolwide and curricu- 
lar goals based on data findings. 

• Prepare second round of interim assessment results for review at common plan- 
ning team meetings. Clarify expectations around using assessment data to support 
student goal-setting. 

• Identify and model best practices for student goal-setting based on October interim 
review process. Identify examples of how the student goal-setting process supports 
schoolwide emphasis on student portfolios and student-led conferences. 

April 

• Finalize materials and protocols for interim assessment data analysis process. 

• Implement interim assessment data review with common planning teams. 

• Coordinate with school leadership team to support analysis of data for program 
improvement review and to set improvement goals based on current data. 

• Coordinate with school leadership team to review high-leverage strategies for 
improvement, including the use of shorter-cycle common assessments. Participate 
in dialogue about how that work might take shape in future given current knowl- 
edge and skills in data analysis. 
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May 

• Work in common planning teams to debrief interim assessment review process. 
Clarify success in helping students set learning goals based on interim data. Reflect 
on this process and review next steps in this work — How do we engage student in 
their performance data to help them understand their strengths and areas where 
they need more focus? 

• Continue to educate and reflect on the work this year in common planning teams 
related to data use. Prepare content for May early-release session to build faculty 
understanding of current data-use practices and to have teachers frame learning 
goals for next year related to data-use practices. 

June 

• Reflect on data team progress this year, clarify what has worked well and where 
additional supports are needed, and prepare report for transformation team regard- 
ing next steps for data team. 

• Develop a calendar of activities and goals for data team during 2012/13 school year. 

• Develop learning goals for data team for summer and upcoming school year, with 
focus on assessment literacy and developing shorter-cycle common assessments. 

• Identify agreed upon summer reading list for data team members. 

Data literacy toolkit topics 

The following list is adapted from a report by Means, Chen, DeBarger and Padilla (2011). 

This resource offers case examples that data coaches or data team members can work 

through to improve their data literacy skills. 

Knowing which data to select 

• Identify the datasets needed to address specific learning or behavioral needs. 

• Understand how to analyze multiple data sets simultaneously. 

Interpreting data accurately 

• Identify data in a table or graph. 

• Understand different data displays (bar graphs, histograms). 

• Manipulate and compare numbers in complex data displays. 

• Understand limitations of sample size and the ability to generalize based on sample 
size. 

• Appreciate the impact of extreme scores on the mean. 

• Understand key assessment concepts, such as test validity and measurement error, 
and the impact these issues have on developing valid inferences. 

Using data for instructional decisionmaking 

• Understand how to differentiate instruction based on data findings. 

• Clarify instructional actions for whole group, small group and individuals based 
on evidence. 

• Understand what the progression of learning looks like for subject-specific content 
at the students’ grade level. 

• Identify appropriate resources for students’ next steps in learning. 
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Engaging in collaborative inquiry 

• Understand and use the inquiry cycle. 

• Work collaboratively to reduce interpretation errors and support logical reasoning 
across the data team. 

• Select tools and protocols to support and focus team dialogue throughout the data 
inquiry cycle. 

Role description of data coach 

The primary role of the school data coach is to work collaboratively with teachers to 
analyze student achievement data and assist teachers in using that data to improve student 
learning. The following is an example of a written role description for a school data coach: 

Under supervision of the school principal, the data coach coordinates, plans, sum- 
marizes, manages, and maintains assessment data. Working collaboratively with 
classroom teachers, site administration, and the leadership team, the data coach 
supports teachers in developing effective data analysis practices and in using the 
findings of data analysis to guide instruction. The data coach works with teachers 
and teacher teams to identify instructional approaches to address identified learn- 
ing needs, provides model lessons that show effective strategies to modify instruc- 
tion, and supports teachers in identifying resources or materials that will best meet 
student-learning needs. Through observation, dialogue, and data analysis the data 
coach contributes to identifying professional learning topics and will work with 
teachers to outline individual learning goals related to data analysis and instruc- 
tional improvement. 

Responsibilities 

• Assist classroom teachers in collecting, analyzing, and using student achievement 
data to guide instruction. 

• Provide model lessons for classroom teachers, using evidence-based teaching. 

• Prepare data presentations for the school principal and staff. 

• Work with the district data coordinator to ensure that teachers can access the 
longitudinal assessment database. 

• Identify site staff development needs using student achievement data. 

• Work with teacher teams to monitor benchmark assessments by student, teacher, 
grade level, standard, and subgroup and plan for extended learning. 

• Coordinate and support data use by response to intervention team. 

• Meet regularly with the principal and leadership team to review progress towards 
goals. 

• Plan, schedule, and conduct site staff development activities aligned with district 
and site goals. 

Knowledge and skills 

• Able to facilitate the use of data analysis protocols during data team meetings. 

• Familiar with the district’s data inquiry model. 

• Skilled in instructional planning. 

• Able to use and analyze multiple types of student assessments and related data. 

• Understands and applies knowledge of adult learning and development. 
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Element V: Examples of leadership policy and guidance 


Example of input to district policy: Principals rewrite their job descriptions 

Upon districtwide development of common short-cycle assessments and school data teams, 
principals asked to rewrite their job descriptions to address changing responsibilities. Prin- 
cipals developed this list and submitted it to district leaders for final revisions. 

• Lead the school data team, set biweekly agendas, and prepare datasets for team 
review. 

• Identify and support school data leaders. 

• Schedule time for teachers to analyze and take action from data. 

• Lead annual analysis of data for school improvement plan. 

• Develop a schoolwide calendar of data-use expectations showing how grade-level 
or department data teams will review evidence through the year. 

• Continue to develop fluency in data-use knowledge and skills. 

• Support teachers in developing skills for both data use and assessment literacy. 

• Work with teachers to use classroom data to set teacher annual learning goals. 

• Model and support developing habits of mind for data use. 

• Create procedures for teams to learn from one another. 

• Create norms of safety and trust when discussing data findings and exploring next 
steps. 

• Support the development and use of short-cycle assessments that are aligned to 
curriculum and measure desired outcomes. 

• Organize data-use technologies. 

• Use current evidence of learning in parent dialogue. 

• Identify funds for and support data-entry staff. 

Building leadership team development activity to establish agreed upon actions for 
principals to strengthen data-use practices 

This team activity can take place at the building leadership team level or with principals 
working together with the district leadership team. 

• Begin by having pairs identify the high-leverage strategies that principals can take 
to strengthen and support data-use practices in the building. 

• Once pairs have a prioritized list, share these lists with the rest of the team. Work 
within the building leadership team to develop a prioritized (and manageable) set 
of actions the principals can take. 

• Clarify what supports or structures are necessary for any new responsibilities. 

• Develop a timeline for implementation. 
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Handout 10.1: Discussion prompts 


These discussion prompts are designed to structure team dialogue that will support iden- 
tifying, clarifying, and crafting written guidance to support enhancing the culture of data 
use in your setting. 

L List the one or two high-priority areas of focus that have been identified in your work 
group or team: 

2. Identify the Culture of Data Use Framework element in which these priority areas 
reside: 

3. Review handout 10.2, which outlines some questions to further explore developing 
guidance, and handout 10.3, which shows examples of guidance. Discuss the questions 
in the framework element(s) under consideration. (8-10 minutes) 

4- Discuss what you know about the issue to be addressed by written guidance. (5-10 
minutes) 

a. How widespread is it? 

b. What do others say about it? 

c. In what ways has it been addressed in the past? 

5. Based on what your team has reviewed in this workshop, discuss examples of effective 
practice in your district or school. Which practices can the team build on to document 
and craft written guidance for this work? (5-10 minutes) 

6. Discuss and review current internal guidance that currently addresses this issue. 

a. What formal guidance has been developed? How has it been disseminated? 

b. What informal expectations are in play? 

7. Ensure agreement about the specific guidance to develop during this time. Take the 
remaining time to begin to craft written guidance in the high-priority area identified. 

8. Outline information about next steps: 

a. Who needs to review this guidance prior to implementation? 

b. How will this guidance be shared with identified reviewers? 

c. What is the internal timeline for finalizing this work? 

d. Who is primarily responsible for moving this work forward? 
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Handout 10.2: Guiding questions for developing policy or 
guidance to support a culture of data use, by framework element 


Participate in the flow of information for evidence use 

Do all members of the district and school have the data they 
need to make effective decisions? 

• What data do we have that can help answer the questions we 
are currently asking about student learning? 

• What improvements to data quality and presentation would 
expand our ability to answer these and other questions? 

Does technology for data use enhance data-use practices by 
educators? 

• What more can be done for system integration to ensure that 
teachers do not have to compensate for limited technology 
use? 


Communicate professional expectations for data use 

Is there a common, shared understanding of how data use 
contributes to a culture of excellence in our district or school? 
• Is there a common, shared understanding that evidence 
of student learning enhances (and does not replace) the 
professional judgment of our educators? 

Do district and school practices align with stated 
expectations? 

Do district and school practices clarify that data use is for 
continuous improvement of instructional practice, rather than 
externally driven for accountability and compliance? 


Provide resources and assistance to make meaning 
from data 

• What are the organizational structures and systems that 
enhance our ability to use data effectively? 

• Is there sufficient and consistent structured time for teachers 
to analyze evidence and explore how to change instruction 
based on that evidence? Is this time scheduled during the 
school day? 

• Do cultural norms allow educators to share real problems of 
practice, both positive and negative? 

Are there opportunities provided to “calibrate” the social 
sense-making processes required of data use, so that 
consistent practices are developed throughout the 
organization? 

Provide professional development on data-use 
knowledge and skills 

Do all members of our district or school have the knowledge 
and skills necessary to make use of the data available to 
them? 

Have we built the internal capacity of all educators to use 
evidence effectively to support learning? 

• Does professional learning take place frequently? Is it 
differentiated to meet the needs of different educators? 

Do educators report that what they learn in professional 
development is immediately useful? 

• Are informal opportunities for professional learning — where 
important knowledge is shared among colleagues to build 
district- and schoolwide capacity — made possible? 

Does professional learning include data use and expanding the 
repertoire of instructional strategies as a result of the data? 


Provide leadership to nurture a culture of data use 

• Are the leadership roles and responsibilities to support evidence use in our schools and at the district level clearly defined? 

• Are there formal opportunities for those in leadership roles to learn and practice how to model and lead effective evidence use? 
Do written role descriptions outline the work that principals or other leaders must do to lead data-use practices? 

Is there agreement about the role that district leaders play to help schools and teachers improve their evidence-use practices? 
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Handout 10.3: Examples of policy and guidance to support a 
culture of data use, by framework element 


Participate in the flow of information for evidence use 

* Written expectations for data use that show which practices 
align with improved practice and document the shift away from 
an accountability-based data-use approach. 

• Written clarification identifying which users of data are meant 
to answer which questions. 

• Calendars and timelines of district data-use expectations by 
grade level. 

° Description of district data systems to clarify functions and 
uses. 

* Description of district data systems that tell how they support 
and align with the everyday work of educators. 

Provide resources and assistance to make meaning 
from data 

• A school calendar documenting scheduled time to analyze 
evidence. 

A common location (computer drive) with written protocols for 
using evidence during team meetings. 

• Written role expectations for data teams with specific team- 
member functions outlined. 

• A schedule with opportunities for calibration. 

Agreed upon norms for data use posted and reviewed at all 
meetings. 

Written documentation of how coaches or instructional 
specialists will support collaborative data-use practices. 

Communicate professional expectations for data use 

• Written guidance about the focus of data use as designed to 
support all students. 

• Written communication highlighting how the focus on all 
students will raise the achievement of struggling learners 
through increased differentiation and personalized student 
supports. 

• Written expectations about annual “products” that include 
student evidence, including written guidance for use of 
evidence at parent-teacher meetings, portfolios, and 
information that is documented across years. 

Provide professional development on data-use 
knowledge and skills 

• An annual schedule of professional learning for data use, 
including formal, informal, large-scale, team-based, and daily 
learning focused on common schoolwide learning goals, 

and including individual (or team) areas of focus for teacher 
learning. 

• Written structure to document teacher learning goals regarding 
using data. 

• Protocol for principals to review teachers’ practices with 
various types of data use (including both schoolwide and 
classroom uses), and to outline next steps in their individual 
learning. 

Provide leadership to nurture a culture of data use 

Job descriptions that capture the role of data-use leaders throughout the district: principals, teacher leaders, data team leaders, 
district leaders. 

• A hiring protocol outlining expectation or activities that show facility with data use. Documentation related to how leaders learn 
data use, with increasing expectations overtime. 
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What’s Happening 
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Research methods for educational settings 


Tools 
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